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In  many  areas  of  scientific  inquiry,  the  phenomena  under  investigation  are 
viewed  as  functions  on  the  real  numbers.  Since  observational  precision  is  limited,  it 
makes  sense  to  view  these  phenomena  as  bounded  functions  on  the  rationals.  One  may 
translate  the  basic  notions  of  recursion  theory  into  this  framework  by  first  interpreting 
a  partial  recursive  function  as  a  function  on  Q.  The  standard  notions  of  inductive 
inference  carry  over  as  well,  with  no  change  in  the  theory. 

When  considering  the  class  of  computable  functions  on  Q,  there  are  a  number 
of  natural  ways  in  which  to  define  the  distance  between  two  functions.  We  utilize 
standard  metrics  to  explore  notions  of  approximate  inference  —  our  inference  machines 
will  attempt  to  guess  values  which  converge  to  the  correct  answer  in  these  metrics. 
We  show  that  the  new  inference  notions,  NVoo,  EXoo,  and  BCoo,  infer  more  classes 
of  functions  than  their  standard  counterparts,  NV,  EX,  and  BC.  Furthermore,  we 
give  precise  inclusions  between  the  new  inference  notions  and  those  in  the  standard 
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inference  hierarchy.  We  also  explore  weaker  notions  of  approximate  inference,  leading 
to  inference  hierarchies  analogous  to  the  EX"  and  BC"  hierarchies.  Oracle  inductive 
inference  is  also  considered,  and  we  give  sufficient  conditions  under  which  approximate 
inference  from  a  generic  oracle  G  is  equivalent  to  approximate  inference  with  only 
finitely  many  queries  to  G. 

Finally,  we  employ  approximation  techniques  from  topology  and  analysis  to 
obtain  a  somewhat  surpising  result  regarding  triviality  in  oracle  inductive  inference 
classes.  A  result  of  Sacks  states  that  if  A  is  non-recursive,  then  the  set  {B  \  A  <t  B} 
has  measure  zero.  Thus,  from  a  measure  theoretic  perspective,  a  nontrivial  (i.e.  non- 
recursive)  oracle  is  comparable  to  few  oracles  containing  more  information  in  the 
partial  order  of  Turing  degrees.  We  show  that  analogous  results  hold  for  the  standard 
notions  of  inductive  inference,  as  well  as  for  the  notions  of  approximate  inference. 
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CHAPTER  1 
INTRODUCTION 

1.1    Basic  Models  of  Inductive  Inference 

In  the  experimental  sciences,  it  is  typical  to  make  generalizations  from  a  finite 
set  of  data,  amending  such  generalizations  as  needed  to  account  for  new  data.  Often, 
the  data  are  collected  at  discrete  time  intervals  and  so  can  be  viewed  as  ordered  in  a 
natural  way.  Also,  the  "experiment"  need  never  end  —  we  may  continue  to  record  a 
new  datum  during  each  time  interval  and  make  a  new  generalization  at  this  time.  It 
is  this  situation  which  we  take  as  our  starting  point.  As  a  further  simplification  we 
view  such  sequences  of  data  as  a  functions  on  the  natural  numbers.  In  the  next  two 
sections,  we  give  an  informal  account  of  the  basic  models  of  inductive  inference,  and 
some  common  variants.  See  Popper  [19]  for  a  philosophical  perspective  on  notions 
of  inductive  inference.  A  more  detailed  mathematical  treatment,  giving  the  formal 
definitions  and  stating  the  major  results  of  inductive  inference,  is  found  in  chapter  2. 

Gold  [10]  was  among  the  first  to  formalize  an  example-based  theory  of  learn- 
ing. The  two  basic  inductive  inference  paradigms  we  informally  describe  below  are 
essentially  due  to  him.  The  reader  is  referred  to  the  excellent  survey  [17]  by  Odifreddi 
for  a  detailed  introduction  to  the  subject  of  inductive  inference  from  a  recursion  the- 
oretic perspective.  In  the  simplest  inference  model,  we  wish  to  define  a  procedure 
which  works  as  a  predictor  for  a  given  class  of  phenomena.  Given  a  sequence  of  data 
(a  partial  record  of  a  phenomenon  in  the  class  under  consideration),  we  wish  to  pre- 
dict what  will  occur  next.  In  this  type  of  inference,  called  next-value  inference,  we 
use  the  sequence  of  data  (c^o, . . .  ,  c?n-i)  collected  prior  to  time  n  to  predict  the  value 
of  the  datum  <f„  which  we  will  collect  at  time  n.  In  view  of  Church's  thesis,  we  will 
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assume  that  we  must  do  so  algorithmically.  That  is,  our  inference  procedure  must 
be  a  recursive  function  —  we  may  not  use  coin  flips,  oracle  consultations,  appeals  to 
higher  beings,  etc.  in  our  attempt  to  predict  the  next  datum. 

We  must  now  consider  the  accuracy  requirements  of  our  prediction  algorithm. 
At  time  t  =  0,  we  are  hardly  in  a  position  to  make  a  reasonable  guess  for  do.  If 
the  underlying  phenomenon  is  complex,  we  may  not  be  in  a  position  to  make  a 
reasonable  guess  for  di\f  at  time  t  =  N,  even  for  large  values  of  A^.  We  will  thus 
allow  our  prediction  algorithm  to  make  some  mistakes,  but  we  will  require  of  the 
algorithm  that  it  is  eventually  always  accurate.  In  other  words,  for  an  algorithm  to 
be  considered  predictive,  it  must  make  only  finitely  many  errors  in  predicting  the 
data.  Henceforth,  if  a  class  C  of  phenomena  is  predicted  by  an  algorithm  M,  we  will 
say  that  M  NV-identifies  C,  or  that  C  E  NV. 

Notice  that  this  restriction  on  the  error  rate  of  our  prediction  algorithm  es- 
sentially forces  us  to  assume  that  the  phenomena  we  are  attempting  to  predict  are 
also  algorithmic,  since  our  prediction  machine  eventually  outputs  the  same  values  as 
those  observed  experimentally.  Thus,  we  impose  the  further  restriction  on  our  class 
of  phenomena  that  they  may  be  viewed  as  recursive  functions;  hence,  our  "world"  is 
the  class  of  recursive  functions. 

Since  we  now  view  all  phenomena  as  "black  boxes"  implementing  algorithms  of 
some  sort,  it  is  natural  to  use  the  data  gathered  from  a  given  "black  box"  to  attempt 
to  guess  its  underlying  algorithm.  Instead  of  simply  guessing  the  next  output,  we  wish 
to  explain  what  is  going  on  inside  the  box.  We  still  have  only  the  data  stream  to  work 
with;  so,  at  time  n  we  will  again  use  the  data  gathered  prior  to  this  time  to  infer  the 
underlying  rule.  As  with  the  predictive  machine,  we  allow  the  explanatory  machine  to 
make  finitely  many  mistakes,  as  long  as  it  eventually  settles  on  an  algorithm  for  each 
phenomenon  in  the  class  it  explains.  If  a  class  C  of  recursive  functions  is  explained 
by  an  algorithm  M,  we  will  say  that  M  EX-identifies  C,  or  that  C  €  EX. 
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Let  us  illustrate  these  concepts  with  a  concrete  example.  The  "phenomena" 
under  consideration  will  be  the  class  V  of  polynomials  on  the  natural  numbers.  Con- 
sider the  following  machine  M,  which  takes  as  input  (/(O), . . .  ,/(n  —  1))  at  stage  n 
(let  L{x)  =  0): 

Stage  n: 

Step  1:  If  L{x)  =  f{x)  for  all  x  <  n,  output  L{n).  Otherwise, 
Step  2:  Form  the  Lagrange  interpolant, 

Set  L  =  L„,  and  output  L{n). 

End  of  Construction. 

Now,  if  the  phenomenon  being  observed  is  a  polynomial  p  of  degree  n,  we 
know  that  the  n  +  1  distinct  data  points  (p(0),...  ,p{n))  uniquely  determine  this 
polynomial,  so  our  algorithm  will  produce  correct  outputs  after  time  n  +  1.  Thus, 
our  algorithm  NV-identifies  V.  It  is  similarly  easy  to  construct  a  max:hine  which 
EX-identifies  V:  at  time  n,  instead  of  outputting  L{n),  output  the  interpolant  L 
itself. 

For  another  example,  consider  the  class  VTZ  of  primitive  recursive  functions. 
Since  we  may  effectively  enumerate  the  members  of  V1Z,  it  is  easy  to  construct  a 
prediction  machine  for  the  class.  Given  inputs  (oq,  . . .  ,an-i)  at  time  n,  we  simply 
search  for  the  first  function  p  in  our  enumeration  which  agrees  with  the  given  data,  i.e. 
such  that  p(0)  =  ao,p(l)  =  oi,  •  ■■p{n  -  1)  =  a„_i,  and  output  p{n).  An  explanatory 
machine  would  use  a  similar  procedure,  but  output  the  function  p  at  stage  n.  At  this 
point,  we  might  guess  that  the  family  NV  of  classes  predicted  by  some  algorithm  is 
identical  to  the  family  EX  of  explanatory  classes.  The  answer  depends  on  what  kinds 
of  algorithms  an  explanatory  machine  is  allowed  to  output  as  its  guesses. 


1.2    Variants  of  the  Basic  Models 

In  a  scientific  explanation  of  the  phenomenon  underlying  observed  data,  it  is 
usually  the  case  that  the  explanation  is  consistent  with  the  data.  In  view  of  this,  we 
may  require  of  our  explanatory  machine  that  the  algorithm  output  at  stage  n  agrees 
with  the  data  given  up  to  this  time.  In  addition,  the  unsolvability  of  the  Halting 
problem  tells  us  that  it  is  in  general  impossible  to  determine  whether  our  output 
algorithms  are  total.  Thus  it  is  also  natural  to  allow  our  explanatory  machine  to 
output  algorithms  which  are  consistent  with  the  available  data  at  time  n,  but  which 
may  not  return  an  answer  for  times  m  >  n  if  we  attempt  to  use  the  algorithm  to 
predict  such  future  values.  If  we  denote  the  family  of  phenomena  classes  which  can 
be  explained  in  this  manner  by  EXcona,  then  EXcona  strictly  contains  NV.  If,  however, 
we  restrict  our  explanatory  machine  to  output  only  total  algorithms,  then  the  two 
families  are  equal. 

Many  other  variations  on  the  basic  model  have  been  explored.  Blum  and  Blum 
[4]  require  that  if  an  explanatory  machine  eventually  stabilizes  on  an  algorithm,  then 
this  algorithm  accurately  describes  the  phenomenon.  This  type  of  inference  is  known 
as  identification  by  reliable  explanation,  and  the  family  of  identified  classes  as  EXrei- 
EXre/-identification  is  more  general  than  EXcona-identification,  but  more  restrictive 
than  EX-identification.  Feldman  [8]  requires  only  that  the  explanatory  machine  even- 
tually output  algorithms  which  are  extensionally  identical;  the  same  algorithm  need 
not  appear  twice,  but  the  algorithms  output  should  all  produce  identical  outputs  on 
identical  inputs.  This  type  of  inference  is  known  as  behaviorally  correct  identification, 
and  it  subsumes  EX-identification. 

One  may  further  relax  the  requirements  for  an  explanatory  machine  by  ac- 
cepting explanations  which  come  "close"  to  describing  the  observed  phenomenon. 
Specifically,  we  may  allow  the  algorithm  upon  which  the  explanatory  machine  finally 
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settles  to  differ  finitely  from  the  observed  data.  Furthermore,  we  may  require  a  uni- 
form upper  bound  on  the  number  of  errors  allowed,  or  we  may  be  a  bit  less  restrictive, 
settling  for  identification  with  arbitrary  finite  errors.  Case  and  Smith  [6]  show  that 
the  hierarchies  of  the  induced  inference  families  are  proper  and  that  behaviorally  cor- 
rect identification  with  arbitrary  finite  errors  is  powerful  enough  to  identify  all  the 
recursive  functions. 

Many  other  variants  appear  in  the  literature.  For  example,  Royer  [22]  defines 
notions  of  probablistic  inference,  and  Case,  Jain  and  Sharma  [5]  define  notions  of 
limiting  inference  which  are  quite  different  from  the  notions  of  approximate  inference 
explored  in  the  sequel.  *.  •  "  ■     ;  .     t'    '}      '.'  .  '       i  *^ 

1.3    Approximate  Inference    <■   ■■.  »     '        -*  ,■ 

Not  all  scientific  inquiry  requires  such  strong  notions  of  inference.  Often,  the 
phenomenon  under  investigation  is  assumed  to  be  a  bounded  continuous  function  on 
the  real  numbers.  After  receiving  only  a  finite  number  of  datum,  the  phenomenon  is 
then  represented  by  a  function  interpolating  the  known  data.  While  the  interpolant 
may  equal  the  underlying  function  at  only  a  few  points,  it  is  nevertheless  viewed 
as  a  reasonable  representation  of  this  function.  In  the  chapters  that  follow,  we  will 
formalize  notions  of  inference  arising  from  this  point  of  view. 

In  chapter  3,  we  introduce  the  concept  of  inference  of  a  recursive  rational- 
valued  function  and  define  an  important  subclass  of  these  functions,  RUC,  which 
cannot  be  inferred  by  the  standard  inference  methods,  but  for  which  a  natural  af>- 
proximation  technique  exists.  This  technique  leads  in  chapter  4  to  new  "approximate" 
inference  paradigms  which  subsume  the  standard  paradigms  and  allow  us  to  easily  in- 
fer the  class  RUC.  In  chapter  5,  we  define  hierarchies  of  approximate  inference  clzisses, 
the  "epsilon"  inference  classes,  which  enable  us  to  infer  more  classes  than  with  the 
approximate  inference  classes,  while  retaining  the  finitistic  flavor  of  the  standard  in- 
ference notions.  In  chapter  6,  we  use  approximation  techniques  from  topology  and 
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analysis  to  obtain  a  surprising  result  regarding  triviality  of  inference  classes.  Finally, 
we  conclude  in  chapter  7  with  some  further  directions  for  research  in  the  area  of 
approximate  inference. 


CHAPTER  2 
PRELIMINARIES 

2.1    Notions  from  Recursion  Theory 

Let  Iq  =  [0, 1]  n  Q.  We  cissume  that  {qo,  Qi,. . .  ,  g„, . . .  }  is  a  fixed  1-1  effective 
enumeration  of  Iq  such  that  qo  =  0,  and  qi  =  I.  [Iq]^'^  will  denote  the  set  of  finite 
sequences  of  rationals.  An  arbitrary  (partial)  recursive  function  can  be  viewed  as 
a  function  on  Iq,  by  interpreting  the  natural  number  m  as  the  m***  rational  in  the 
enumeration  {^nlnew  More  formally,  we  could  define  the  rational  interpretation  /q 
of  a  (partial)  recursive  /  by  fQ{qm)  =  qn  iff  /('^)  =  In  this  framework,  we  could 
carry  out  all  basic  recursion  theory,  proving  the  enumeration  theorem,  the  and 
fixed  point  theorems,  and  so  forth.  In  the  sequel,  we  will  dispense  with  most  of 
this  formalism,  preferring  to  use  notation  to  indicate  the  types  of  functions  under 
consideration. 

Let  (co, . . .  ,  a„_i)  denote  the  usual  coding  of  a  finite  sequence  of  natural  num- 
bers (or  rationals)  by  a  natural  number.  For  a  function  /  and  natural  number  n,  f\^ 
denotes  (/(O),...  ,/(n))  (or  {/{qo),-.  -  ,/(9n)))-  We  denote  finite  binary  sequences 
by  (7,  r.  In  this  context,  \a\  denotes  the  length  of  the  a.  If  a  is  an  initial  segment  of 
r  (or  5  €  2'^),  we  denote  this  hy  a  ~<  t  {a  ~<  B).  We  will  use  the  following  to  denote 
intervals  in  2'^:  I{a)  =  {B  e  2"^  \  <t  ^  B}. 

{<^o,  4>i,  -  ■  ■  ,  •  •  •  }  enumerates  the  the  partial  recursive  functions,  and  (j>  de- 
notes an  arbitrary  partial  recursive  function.  {$0j  ^i)  •  •  •  ,  ^n?  •  •  •  }  enumerates  the 
corresponding  functions  on  Iq,  with  $  denoting  an  arbitrary  partial  recursive  func- 
tion with  domain  and  range  contained  in  Iq.  M  (with  adornments)  will  denote  a 
(standard)  recursive  Turing  machine,  thought  of  as  a  machine  which  infers  a  class 
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of  functions.  Such  machines  M  will  usually  be  used  in  the  context  of  the  standard 
inference  classes,  operating  on  sequences  of  natural  numbers,  and  producing  natural 
numbers  as  outputs.  We  will  also  use  them  in  the  context  of  standard  inference  of 

oo 

functions  on  /q.  We  will  often  use  the  notation  "Vx"  in  this  context  to  mean  "for  all 
but  finitely  many  x" .  G  (with  adornments)  denotes  a  recursive  Turing  machine  with 
inputs  from  [/q]*^'^  and  outputs  in  Iq  or  N,  depending  on  whether  its  guesses  are  to 
be  function  values,  or  indices  of  functions. 

REC  will  denote  the  standard  (total)  recursive  functions  (i.e.  those  recursive 
on  N).  QREC  will  denote  the  class  of  total  recursive  functions  on  /q;  we  will  also  call 
these  the  recursive  rational-valued  functions.  We  shall  denote  the  subclass  of  recursive 
{0,  l}-valued  functions  by  QSET.  RC  denotes  the  recursive  rational  functions  which 
are  continuous  on  Iq,  and  RUC  denotes  those  which  are  uniformly  continuous  on 
Iq.  Since  Iq  is  neither  compact  nor  connected,  continuity  does  not  imply  uniform 
continuity,  so  that  RUC  C  RC.  Note  that  we  may  embed  REC  into  RUC  hy  mapping 
/  to  /q,  and  then  mapping  /q  to  /q  defined  by  /q(0)  =  0,  and  for  n  >  0,  fQ{l/n)  = 
/Q(gn_i)/n,  linearly  interpolating  these  values  for  other  rationals. 

See  Odifreddi  [16],  Rogers  [21]  or  Soare  [24]  for  developments  of  basic  recursion 
theory.  All  notions  from  measure  theory,  real  analysis,  and  topology  can  be  found  in 
Halmos  [11],  Royden  [20],  and  Munkres  [15],  respectively. 

2.2    Inductive  Inference  ,        Ij,.'  ^ 

We  give  a  brief  summary  of  the  standard  inductive  inference  classes  and  the 
theorems  relating  them.  The  three  basic  notions  of  inductive  inference  are  as  follows: 

Definition  2.2.1.  A  class  C  of  recursive  functions  is  next-value  identifiable  {C  € 
NV)  if  there  is  an  M  :  [w]^'^  — >  u>  such  that  for  every  f  EC, 

oo 

Vn/(n)  =  M((/(0),...,/(n-l))). 


Thus,  an  JVV-inference  machine  M  tries  to  correctly  guess  the  sequence  {/(O), 
/(I),...},  making  only  finitely  many  errors.  We  also  say  that  M  NV-identifies  C, 
and  that  C  €  NV  via  M. 

Definition  2.2.2.  A  class  C  of  recursive  functions  is  explanatorily  identifiable  {C  € 
EX)  if  there  is  a  M  :  [w]*"'^  — y  u  so  that  for  every  f  E  C,  there  is  an  index  e  of  f 
such  that 

oo 

Vn  M((/(0),...,/(n-l)))  =  e. 

Thus,  an  EX-inference  machine  M  tries  to  settle  on  a  correct  index  for  the 
input  /,  making  only  finitely  many  errors.  Gold  [10]  also  introduces  the  notion  of 
explanatory  consistency: 

Definition  2.2.3.  A  class  C  of  recursive  functions  is  identifiable  by  consistent  ex- 
planation (C  €  EXcona)  if  C  €  EX  via  M,  and  for  all  n, 

<^A/((/(0),...,/(n-l)))|„  = /!„• 

Feldman  [8]  further  weakens  the  notion  of  EX-inference  as  follows:  ^. 

Definition  2.2.4.  A  class  C  of  recursive  functions  is  behaviorally  correctly  identifi- 
able {C  €  BC)  if  there  is  a  M  :  [u]^'^  — >  u  so  that  for  every  /  €  C, 

oo 

Vn  (f>M({m  /(n-1)))  =  /• 

Thus,  a  BC-inference  machine  M  must,  after  finitely  many  stages,  output  only 
indices  of  the  input  function  /.  However,  unlike  the  EX  case,  we  may  output  any 
index  of  /,  infinitely  often  changing  our  mind  about  which  index  to  output. 

The  notions  PEX  and  PBC  are  similar  to  EX  and  BC,  except  that  M  may 
only  output  indices  of  total  functions.  Theorems  of  Blum  and  Blum  [4],  Barzdin  [3], 
Case  and  Smith  [6],  Gold  [10],  and  Harrington  [12]  yield  the  following  hierarchy  of 
inference  classes: 

Proposition  2.2.5.  NV=  PEXC  EX^^  C  EXC  BC. 
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Blum  and  Blum  [4],  and  Case  and  Smith  [6]  introduce  the  concept  of  inference 
with  anomalies: 

Definition  2.2.6.  If  /  is  total,  we  say  that  0e  is  an  n-variant  of  /  (<^e  ~"  /)  if 

|{x|<^e(x)tor0e(x)4//(x)}|<n. 

We  say  simply  that  (f>e  is  a  variant  of  /  {4>e  — *  /)  if  4>e  is  an  n-variant  of  /  for  some 

■   ■      :  ,      * .    ■  f    -  ■  ■ 

Definition  2.2.7.  A  class  C  of  recursive  functions  is  explanatorily  identifiable  with 
n  errors  [C  €  EX^)  if  there  is  an  M  :  [jj]^'^  — >  uj  so  that  for  every  f  E  C,  there  is 
an  index  e  such  that  <^e  — "     and  ,  ^« . , ,  ^ 

oo 

Vn  M((/(0),...  ,/(n-l)))  =  e. 

Definition  2.2.8.  A  class  C  of  recursive  functions  is  explanatorily  identifiable  with 
finitely  many  errors  {C  €  EX*)  if  there  is  an  M  :  [w]*''^  — >  OJ  so  that  for  every  f  EC, 
there  is  an  index  e  such  that  (f>e  c^*  /,  and 

oo 

VnM((/(0),...  ,/(n-l)))  =  e. 

Note  that  EX^  =  EX.  The  aforementioned  authors  have  shown  the  following: 
Proposition  2.2.9.  For  all  n, 

•  EX"  CEX*  (in  fact,  U„EX"  C  EX*),  and 

•  EX"  C  EX"+^ 

Case  and  Smith  [6]  show  that  EX*  C  BC.  They  also  extend  the  concept  of 
inference  with  anomalies  to  BC  —  BC"  and  BC*  are  defined  similarly  to  their  EX- 
counterparts.  The  previous  proposition  also  holds  for  BC"  and  BC.  Harrington  [12] 
shows  that  BC  is  as  weak  a  notion  of  inference  as  possible,  in  the  sense  that  REC  E 
BC. 
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We  thus  have  the  following  hierarchy  of  inference  classes: 

NV=PEXCEXcons 

cexcex^ceX^c...c  u„ex"  c  ex* 

CBCCBC^CBCl^C...gUnBCr^BCr. 
In  the  sequel,  we  shall  see  that  the  notions  of  approximate  inference  yield  a  hierarchy 
which  is  no  longer  linear.  "         ;         >  ■  .      W-:  { 

2.3    Inference  with  Oracles  •  _  * 

Fortnow  et  al.  [9],  and  Kummer  and  Stephan  [14]  exanmine  the  concept  of 
oracle  inductive  inference,  in  which  the  inference  machine  M  is  allowed  queries  to  an 
arbitrary  set     C  N. 

Definition  2.3.1.  We  say  that  an  oracle  Turing  machine  (O.T.M.)  M^^  is  categorical 
if  for  every  A  C  N,  M"^  is  total  (in  the  sequel,  we  shall  usually  identify  such  A  with 
its  characteristic  function). 

We  give  the  definitions  of  iVV[/l],  iVV[yl*]  here.  The  definitions  for  EX[A], 
EX[y4*],  BC[v4],  and  BC[/1*]  are  obtained  from  the  definitions  of  EX  and  BC  similarly. 

Definition  2.3.2.  Let  A  C  N.  A  class  C  of  recursive  functions  is  next-value  iden- 
tifiable from  A  {C  ^  NV[A])  if  there  is  a  categorical  O.T.M.  M^^  so  that  for  every 

fee, 

Vn/(n)  =  M^((/(0),...,/(n-l))). 

Definition  2.3.3.  Let  ^4  C  N.  A  class  C  of  recursive  functions  is  next-value  identi- 
fiable from  A  with  finitely  many  queries  (C  €  iVV[y4*])  if  there  is  a  categorical  O.T.M. 
M 0  so  that  C  e  NV[A]  via  M^,  and  for  each  f  eC,  there  is  an  n  so  that  /  is  inferred 
by  M"*"^"  

For  oracle  inference  machines,  the  fundamental  task  is  to  determine  which 
oracles  yield  no  increase  in  inference  power  and  which  ones  allow  us  to  infer  all  of 
REC. 
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Definition  2.3.4.  We  say  that  A  is  NV-trivial  if  NV[A]  =  JVV,  and  NV-omniscient 
if  REC  €  iVVfyl].  We  define  these  notions  for  other  inference  criteria  similarly. 

K  denotes  the  Halting  problem,  K  =  {{x,y)  \  (f>x{y)  i}-  It  is  well-known  that 
K  is  EX-omniscient.  An  O.T.M.  which  EX-identifies  REC  using  K  may  be  defined  as 
follows  (on  input  /  G  REC): 

Stage  n: 

Case  1:  There  is  an  e  <  n  such  that  </>e(y)  4-=  f{y)  for  all  y  <  n  (use  K  to 
determine  this).  Output  the  least  such.     '  ' 

■      ;       !  i 

Case  2:  No  such  e  exists.  Output  n. 

End  of  Construction. 

As  another  example,  consider  -  ,  j    i  .  '  ■ 

TOr={e|Vy0e(y)i}. 

It  is  easy  to  see  that  REC  G  EX[TOT*\.  The  machine  to  infer  REC  is  as  follows 
(input  /  €  REC,  and  let  cq  =  0): 

Stage  n: 

Case  1:  Cq  €  TOT,  and  for  all  y  <  n  (f>eo{y)  =  /(y)-  Output  cq.    .  , 

Case  2:  There  is  an  e  <  n  such  that  e  €  TOT,  and  for  all  y  <  n  <f>e{y)  =  /(y)- 
Output  the  least  such  e,  and  set  cq  =  e.  "  ' 

Case  3:  No  such  e  exists.  Output  cq. 
End  of  Construction. 

Definition  2.3.5.  C  €  2'^  is  generic  if  for  each  I>i  W  C  {0, 1}*,  either 
•  {3a  ^G)  (Wry  a)  t^W,ot 
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Following  Fortnow  et  al.  [9],  we  use  use  the  notation  Q{A)  to  mean  that  either 
A  is  recursive,  or  A  <t  K  and  A  =t  G  for  some  generic  G. 

The  main  result  in  Fortnow  et  al.  [9]  is  that  EX[A]  =  EX  <^  g{A)  and  BC[A]  = 
BC  <^  Q{A).  The  crux  of  the  argument  is  to  show  that  for  generic  G,  EX[G*]  =  EX[G\ 
and  that  BC[G*]  =  BC[G]. 


CHAPTER  3 

INFERENCE  OF  RECURSIVELY  CONTINUOUS  FUNCTIONS 
3.1    The  Recursively  Continuous  Functions 

We  recall  the  definition  of  a  recursively  continuous  function  on  the  rationed 
unit  interval  Iq: 

Definition  3.1.1.  /  is  recursively  continuous  if  /  is  recursive,  and  uniformly  con- 
tinuous w.r.t.  the  standard  metric  on  Q.  We  denote  the  class  of  all  such  functions  by 
RUC. 

There  are  other  versions  of  continuity  which  we  could  consider  in  place  of  uni- 
form continuity  in  the  above  definition.  Note  that  any  uniformly  continuous  function 
on  /q  has  a  unique  uniformly  continuous  extension  to  [0, 1].  However,  an  /  which  is 
continuous  on  Q,  but  not  uniformly  so,  may  have  no  continuous  extension  to  all  of 
[0, 1].  Thus,  with  our  present  definition,  we  may  identify  /  €  RUC  with  its  unique 
continuous  extension  to  [0, 1].  We  could  also  consider  a  more  standard  definition  of  a 
"recursively  continuous"  real  function.  For  example,  Cenzer  and  Remmel[7]  define  a 
real-valued  /  to  be  recursively  continuous  if  there  is  a  uniformly  recursive  sequence 
{/„}  of  functions  on  Q,  and  a  recursive  modulus  of  convergence  e  such  that  for  any 
r  e  R,  and  9  €  Q,  if  m  >  e(n),  then 

e[n)  n 
Using  this  definition,  for  a  rational  input  r,  we  could  then  approximate  /(r)  to  within 

any  desired  tolerance.  However,  this  does  not  allow  us  to  directly  compute  /(r),  even 

if  /(r)  is  rational  as  well.  Furthermore,  functions  which  are  recursively  continuous 

in  this  sense  are  not  necessarily  recursive  in  the  usual  sense,  and  in  particular,  may 

not  be  in  the  class  RUC  defined  above.  Since  we  will  be  considering  only  recursive 
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functions  in  the  sequel,  we  allow  our  definition  of  RUC  to  stand.  See  also  Pour-El 
and  Richards  [18]  for  alternate  definitions  of  recursive  continuity. 

In  the  experimental  sciences,  and  in  many  engineering  applications,  the  ele- 
ments of  a  class  of  phenomena  under  consideration  are  assumed  to  be  continuous. 
Thus,  it  is  natural  to  consider  the  effect  of  restricting  the  usual  inference  notions  to 
such  a  class.  In  this  chapter,  we  will  show  that  RUC  is  a  domain  in  which  all  of  the 
standard  inference  notions  can  be  formulated  and  in  which  the  standard  inference 
hierarchy  given  in  the  last  section  is  preserved.  This  is  in  some  sense  a  metisure  of 
the  nontriviality  of  RUC:  if,  for  example,  we  were  to  restrict  inductive  inference  to 
the  class  of  polynomials  over  N  or  to  the  primitive  recursive  functions,  the  inference 
hierarchy  would  collapse.  We  first  show  some  basic  properties  of  RUC. 

Note  that  RUC  ^  QREC:  if  A  C  a;  is  finite,  then  xa  is  recursive,  but  not 
continuous.  The  class  RUC  is  H": 

e  €  RUC<^  e  €  TOT  k  VmBnVA;,/ 

\qk  -  9/1  <  1/n      \(f)e{qk)  -  M<li)\  <  ^1^- 

In  fact,  RUC  is  Ha-complete,  as  we  will  show  in  the  next  theorem. 
Theorem  3.1.2.  The  class  RUC  is  n^-complete. 

Proof.  We  first  construct  a  computable  irrational  r  and  a  computable  sequence  of 
irrationals  {r„}„gu,  s.t.  0  <  tq  <  r2  <  •  •  •  <  r,  1  >  ri  >  ra  >  •  •  •  >  r,  with  r2„  — ^  r 
and  r2n+i  r.  Let  r  =  .1010010001000  ••  •  (clearly,  r  is  computable).  Now,  define 
the  sequence  {r^}fcga>  by  letting  r2k  be  the  irrational  formed  by  replacing  the  A;"*  1  in 
the  binary  expansion  of  r  by  0,  and  letting  r2jt+i  be  the  irrational  formed  by  replacing 
the  k^^  0  in  the  binary  expansion  of  r  by  1.  Then  {rk)kef^  is  clearly  computable.  Let 
Eq  =  [0,ro),  E2k  =  [r2k-2,r2k),  Ei  =  (ri,l],  and  Eik+i  =  (r2jfc+i, rajt-i]. 

Given  a  set  A,  we  shall,  uniformly  in  a,  construct  a  recursive  function  /„ 
s.t.  fa  €  RUC  iff  a  €  A.  The  essence  of  the  construction  is  as  follows.  If  A  is  H", 
then  there  is  a  recursive  relation  R  s.t.  a  €.  A      WiSjVk  R{i,j,k,a).  We  will  define 
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a  recursive  map  fa  :  /q  — >^  Iq  which  is  uniformly  continuous  if  and  only  if  a  €  ^. 
On  the  interval  Ei,  fa  consists  of  a  series  of  "sawtooths"  of  height  1/2',  possibly 
interspersed  with  "gaps"  (sub-intervals  on  which  fa  is  identically  zero).  To  make  fa 
recursive,  we  define  fa  on  each  rational  qk  at  no  later  than  stage  k  of  the  construction. 
Now,  if  q  is  the  first  rational  enumerated  into  we  let  =  0  on  n  Q.  Initially, 
we  will  try  to  satisfy  the  relation  R  in  Ei  with  j  =  0.  Since  j  will  depend  on  i, 
we  will  denote  j  in  the  i^^  interval  by  j,-.  A  "sawtooth"  is  added  in  the  interval  Ei 
whenever  we  find  a  new  witness  that  a  ^  A,  that  is,  a  new  q  =  qk  enumerated  into 
Ei  such  that  {3k  <  k)-'R{i,ji,k,a).  In  this  case,  if  fa  is  not  already  defined  at  qk,  fa 
is  extended  by  a  "sawtooth"  in  Ei  from  the  greatest  rational  previously  enumerated 
into  the  interval  to  the  new  point  qk,  and  j,  is  incremented  (since  the  current  ji  c«in 
no  longer  witness  that  a  €  A).  If  no  such  witness  is  found,  i.e.  if  (VA;  <  k)R{i,ji,  k,  a), 
a  "gap"  is  added  —  in  other  words,  /„  is  extended  by  the  zero  function  to  qk- 

Now,  if  a  ^  A,  then  there  will  be  at  least  one  I'o  such  that  to  each  j  there  will 
correspond  a  kj  such  that  -^R{iQ,j,kj,a).  Since  D  Q  is  dense  in  Eig,  for  any  j 
there  will  be  a  qk  enumerated  into  Ei^  with  kj  <  k,  and  with  /„  as  yet  undefined  at 
qk.  Thus,  fa  will  have  an  infinite  number  of  "sawtooths"  of  the  same  height  on  Ei^. 
But  then  lim,_vr;^  does  not  exist,  so  that  fa  will  is  not  uniformly  continuous,  hence 
not  recursively  continuous.  On  the  other  hand,  if  a  €  A,  for  each  i,  there  will  be  a 
stage  Ni  in  our  construction  after  which  ji  is  no  longer  incremented  (j,  will  be  the 
least  j  for  which  R(i,j,k,a)  holds  for  all  k).  Thus,  for  q  =  qk  enumerated  into  Ei 
after  stage  A^,-,  {\/k  <  k)R{i,ji,  k,  a)  will  hold.  So,  if  a  €  v4,  /„  will  be  zero  for  q  >  q^^ 
in  each  Ei.  Thus,  /„  is  continuous  at  every  rational  in  Ei,  and 

•  lim^_>.r,  =  0  at  each  r,-,  and 

•  lim^_^r  =  0,  since  the  "sawtooths"  in  each  Ei  decrease  in  height  to  zero  as 
r,-  — >  r. 
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We  now  give  a  more  formal  algorithm  for  fa-  The  function  s  in  the  algorithm 
will  provide  the  precise  "sawtooth"  needed  at  stage  k.  We  define  s  by  . 

,.^>,_/2x,  if0<x<l/2, 
^  >  ~\  -2x  +  2,  if  1/2  <  X  <  1, 

and  scale  s  to  suit  our  needs  at  any  particular  stage. 

Stage  k:  First,  we  must  determine  to  which  Ei  the  rational  belongs.  This  is  effec- 
tive, since  the  {r,}  are  uniformly  computable.  Once  this  is  determined,  we  determine 
/a  (g*;)  as  follows:  :     -   .  ■'■c.'i: 

Case  1:  qk  is  the  first  rational  enumerated  into  Ei.  Then  set  fa{x)  =  0  for  all 
X  <  qk  in  Ei.  Set  X,  =  qk,  and  set  j,  =  0. 

Case  2:  qu  <  x,.  In  this  case,  do  nothing,  since  /o(9jt)  is  defined  at  a  previous 
stage. 

Case  3:  qk  >  x,-  and  (VA;  <  k)  R{i,ji,k,a).  Then  set  fa{x)  =  0  for  all  x  in  the 
interval  [x,-,gjfc].  Set  x,-  =  qk- 

Case  4-'  qk  >      and  {3k  <  k)  -<R{i,ji,k,a).  Then  extend  /„  on  the  interval 
[x,-,gA:]  using  the  "sawtooth"  function: 

1     /  X  —  X,  \ 
2^^  \qk-XiJ  ' 

Set  X,-  =  qk,  and  increment  j,-. 

The  function  fa  is  uniformly  recursive  in  a,  so  we  have  defined  a  binary 
recursive  function  ^|^{a,n).  Thus,  by  the  Theorem,  there  is  a  recursive  /  s.t. 
0/(a)(«)  =  i>{a,n).  But  then,  /  is  the  desired  reducibility,  for  if  a  6  A,  then  for  each 
I,  the  last  case  above  is  never  satisfied  beyond  some  stage  A:,-.  Thus,  <^/(o)  is  eventually 
zero  in  each  interval,  so  </>/(a)  is  continuous  on  /q.  On  the  other  hand,  \{  a  ^  A,  then 
there  is  an  i  s.t.  the  last  case  comes  up  infinitely  often,  and  so  lima;_^ri  4>s(a){x)  does 
not  exist,  whence  (t>j(a)  is  not  continuous  (see  figure  3.1).  □ 
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Figure  3.1:  The  construction  of  /. 


In  the  sequel,  it  will  be  useful  to  encode  a  recursive  function  into  a  recursively 
continuous  one.  In  fact,  we  shall  produce  an  effective  embedding  e  of  REC  into 
RUC,  allowing  us  to  perform  the  encoding  in  a  uniform  way.  Furthermore,  it  will 
be  easy  to  decode  the  values  of  the  recursive  function  from  the  RUC  function  into 
which  it  has  been  embedded.  First,  note  that  we  may  effectively  obtain  a  strictly 
decreasing  sequence  {qj}j^^  from  our  enumeration  of  /q,  such  that  qo  =  1  and 

limj_^oo9i  —  0-  Now,  given  /  €  REC,  we  define  /  =  £(/)  on  the  {qj}j^^  by 

and  by  linear  interpolation  on  rationals  between  these  points.  Finally,  we  set  /(O)  =  0. 
Clearly,  the  map  e  is  one-to-one,  and  for  any  /  G  REC,  f  is  recursively  continuous. 
e  also  has  an  effective  inverse,  since  qj^j)  =  2^f{qj).  We  next  consider  the  effect  of 
restricting  the  standard  notions  of  inference  to  the  class  RUC. 
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3.2    Next  Value  Inference  of  RUC 

Let  us  consider  the  inference  class  NV  ,  which  is  simply  the  clciss  NV, 
restricted  to  the  recursively  continuous  functions.  As  mentioned  in  the  Introduction, 
REC  i  NV.  We  now  prove  the  analog  for  NV^^^. 

Theorem  3.2.1.  RUC  ^  NV^^^. 

Proof.  Given  a  recursive  M,  we  will  construct  a  recursively  continuous  /  for  which 
M  is  not  a  next-value  function.  We  will  give  the  value  of  /  on  Qn+i  by  linear  inter- 
polation, using  the  points  (^o, /(9o)),  (gi, /(9i)),  •  •  •  ,(9n,/(9n))-  First,  let  /(O)  =  0, 
/(I)  =  1.  Now,  suppose  that  f{qo),f{qi),-  •  •  ,fiqn)  have  been  determined,  and  that 
the  set  {qo,  q\,. . .  ,  g„}  is  ordered  as  0  =  <  ^t,  <  . . .  <  qk„.  Let  /i„,,(a;)  denote  the 
linear  interpolation  of  the  points  {qm,  fiqui))  and  (9ni+u/(9ni+i))?  and  /i„  the  func- 
tion obtained  by  "pasting"  the  hn,i  together.  If  M{{f{qo), . . .  ,  /(^n)))  does  not  equal 
hniqn+i),  set  f{qn+i)  =  /in(9n+i).  If,  howevcr,  M{{f{qo),...  ,/(9n)))  =  /i„(gn+i), 
set  f{qn+i  =  /in(gn+i)  +  ^kr  if  hn{qn+i)  +  2^  <  1,  otherwise,  set  /(9„+i)  = 

hn{qn+l)  — 

The  function  /  is  recursive,  since  f(qn)  is  defined  at  stage  n.  Note  that  |/i„(x)  — 
hn+i{x)\  <  2~"~^  for  each  n  and  x,  so  that  the  real-valued,  continuous  functions 
{hn}  converge  uniformly  to  a  real- valued,  continuous  function  h  on  [0,1].  Then  h 
is  uniformly  continuous  on  [0,1].  Thus,  since  /  is  just  the  restriction  of  h  to  /q,  / 
is  uniformly  continuous  on  /q,  i.e.  /  €  RUC.  By  construction,  M  is  not  a  next- 
value  function  for  /.  Since  M  was  arbitrary,  we  have  shown  that  no  recursive  Turing 
machine  can  be  a  next-value  function  for  all  of  RUC,  which  proves  the  theorem.  □ 

3.3    Inference  of  RUC  by  Consistent  Explanation 

We  now  consider  ExJ^^,  the  restriction  of  EXcons  to  RUC.  The  proof  that 
RUC  ^  EX^g^j    uses  a  method  similar  to  the  one  used  in  the  proof  by  Odifreddi  [17] 

that  REC^  EXcon,- 
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Theorem  3.3.1.  R[/C^  f:x£^^. 

Proof.  We  will  assume  that  M  witnesses  that  RUC  €  EX^„,  and  obtain  a  contra- 
diction by  constructing  a  recursively  continuous  function  /  in  a  manner  which  pro- 
hibits M  from  stabilizing  on  any  index  for  /.  We  will  accomplish  this  by  constructing 
a  sequence  {^nj  so  that  M{{f{qo),...  ,/(9nJ))  ^  M{{f{qo),...  , /(g,„^^, ))),  thus 
ensuring  that  ;  '  ■ 

lim  M({/(9o),...,/(gn))) 

n—ioo  . 

does  not  exist.  As  we  have  done  before,  we  will  construct  a  sequence  of  real-valued 
functions  {hn}  such  that  the  limiting  function  h  is  continuous,  with  /  being  the 
restriction  of  h  to  the  rationals.  ho  will  be  identically  zero.  At  stage  n  we  extend  /i„ 
past  Qn  in  two  different  ways,  with  h  denoting  the  extension  which  agrees  with  /»„, 
and  h'  denoting  a  function  obtained  by  perturbing  hn  past  q„  by  a  "hat"  function 
of  height  2~".  The  restrictions  to  Q  of  A  and  h'  will  both  be  recursively  continuous, 
and  different,  so  M  must  eventually  produce  different  indices  for  them.  We  can  then 
effectively  find  Uk  =  the  least  such  n  such  that  M  produces  different  indices.  We 
then  set  /i„+i  equal  to  whichever  of  h,h'  yielded  an  index  different  from  the  index 
produced  by  M  at  stage  njt-i,  and  let  f{qn)  =  hn+i{qn).  Thus  /  will  be  recursive, 
and  uniformly  continuous,  but  M  will  not  identify  it,  showing  that  RUC  ^  EX^^  . 
We  now  give  the  details  of  the  construction.  Let 

qk  =  mdLK{qo,qi,...  ,qk}\{l}. 

At  stage  k  in  the  construction,  there  are  two  possible  cases.  If  f{qk)  is  already  defined, 
do  nothing.  On  the  other  hand,  if  f{qk)  is  not  yet  defined,  we  extend  f  continuously 
past  qk-i  in  two  different  ways,  by  h  and  h'  as  described  above.  For  notational 
convenience,  set  =  0,  n_i  =  0,  and  let  s  denote  the  "sawtooth"  function  of 
Theorem  3.1.2.  Also,  let  =  M{{f{qo),...  ,f{qk-i))).  We  begin  our  construction 
with  /(O)  =  /(I)  =  0. 
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Define  the  functions  hk,h,h'  at  each  stage  k  such  that  f{qk)  is  undefined  as 
follows  (at  stages  k  where  f{qk)  is  already  defined,  do  nothing):  .  ?  ; 

{ >     -  - 
hk-i{x),         if  qk  =  qk-i  OT  X  <  Qk,  '■  - 

'   otherwise.  ^  ,         ^  ^, 

Now,  we  can  effectively  find  n^;  >  nfc_i  such  that   '      ^    '     '  ,  m- 

ai  =  M((%o), . . .  ,  %„ J))  ^  M((/i'(9o), . . .  , /»'(gn J))  =  02. 

Let  hk  =  h  if  ai  ^  e^,  and  /i'  otherwise.  Let  /(g^)  =  hk{qk),  thus  /  is  recursive.  As 
in  the  previous  construction,  we  see  that  \hk{x)  —  hk+i{x)\  <  2~*"*  for  each  A;.  Thus, 
if  we  set  h{x)  =  Yimk-^oohkix),  then  h  is  the  uniform  limit  of  continuous  functions, 
so  h  is  continuous,  hence  uniformly  continuous  on  [0, 1].  But  /  is  just  the  restriction 
of  h  to  Iq,  so  that  /  is  also  uniformly  continuous  on  /q,  whence  /  €  RUC.  By 
construction,  M  does  not  identify  /,  which  proves  the  theorem.  □ 

As  noted  previously,  if  we  modify  the  definition  of  EXcom  so  that  the  inference 
machine  M  must  always  output  the  index  of  a  total  recursive  function  (calling  the 
new  inference  notion  PEXcons),  we  have  PEXcona  =  NV.  Restricting  to  RUC,  it  is 
easy  to  show  that  PEX^^^^  =  NV^^^. 

Theorem  3.3.2.  PEX?i!^=  NV^^^. 


cons 


Proof.  Let  M  iVV^^ ^-identify  C.  For  any  sequence  a  =  (ao,  •  •  •  jOn-i),  let  h{a) 
be  the  least  i  <  n  s.t.  Vi  <  j  <  n  M((ao, . . .  ,aj_i))  =  aj,  if  one  exists;  otherwise  let 
h{a)  =  n.  Let 

{Ux,        if  a  is  a  sequence  of  length  n  and  x  <  h{a), 
M (o),   if  a  is  a  sequence  number,  and  x  >  h(a), 
0,  otherwise. 

By  the  theorem,  there  is  a  recursive  M  s.t.  <^X/(e)(^)  —  V'(e)3^)-  Note  that 
M(n)  is  a  recursive  index  for  all  n,  since  is  a  recursive  function  of  two  variables. 
Furthermore,  if  /  €  C,  there  is  an  iV  s.t.  M  agrees  with  /  forn  >  A^.  If  N  is  minimal, 
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then  /i((/(0),  /(I), . . .  ,  f{n)))  =  for  all  n  >  N,  so  that  M((/(0), . . .  ,  /(n)))  is  an 
index  for  /  at  each  stage  n  >  N.  Thus,  we  need  only  to  ensure  that  these  indices 
stabilize  in  order  that  C  €  PEX^„,  .  To  accomplish  this,  at  stage  n  we  simply  output 
the  first  e  =  (/(O), . . .  ,f{k))  <  n  such  that  /  agrees  with  <^x/(e)  *° 

In  the  other  direction,  if  M  PEX^„^  -identifies  C,  then  at  stage  n+1  we  output 

^MUMJM  /(9n))(«+l)-  This  is  recursive,  since  M(/(go),/(gi),---  JM)  €  TOT, 

and  it  iVy^ ^^-identifies  C,  since  if  f  E  C,  then  M  stabilizes  on  an  index  of  /  after 
some  stage  N,  so  that  for  all  n  >  A'^  </'M(/(?o),/(9i).-,/(«Ar))("  +  1)  =  /("  +  ^ 

Recall  that  NV  C  EXco„3.  We  note  that  the  analogous  inclusion  NV^^^  $ 
EX^J^^  for  the  relativized  inference  types  holds  as  well.  The  proof  that  NV  C  EXcon* 
given  by  Odifreddi  [17]  does  not  easily  relativize.  Our  proof  will  make  use  of  the 
encoding  function  e  :  REC  — >  RUC  given  above.  For  C  C  REC,  let  C  denote  the 
set  {e(/)|/  e  C}.  Then  jVV^^^  C  Ex£„^^  is  an  easy  corollary  of  the  lemmas  that 
follow. 

Definition  3.3.3.  For  any  M,  cr  =  (oo,...  ,a^_i),  we  define  Sm,^  on  REC  by 
recursion  as  follows 

{M((ao,...  ,a„_i)),  if  n<m,  / 

M((ao,--- ,am-i,'S'A/,(r(m),..., 
SM,a{n  —  1)),  otherwise. 

Lemma  3.3.4.  CeNV^Ce  NV^^^. 

Proof.  Suppose  M  A'^V-identifies  C,  and  let  h  £  C.  At  stage  n  + 1,  we  are  given  the 
values  h{qo),h{qi), . . .  ,/t(g„)  from  which  to  make  a  prediction  for  h{qn+\).  Among 
these  values  will  be  h{qo),h{qi), . . .  ,  h{qk„)  for  some  A;„.  But  h  =  e(/)  for  some  f  €^C, 
so  we  may  effectively  find  f{i)  from  h{qi),  using  the  construction  of  e.  Now,  let  /„  = 

5m,(/{o)  j(kn))-  Then,  as  a  guess  for  /i(9„+i)  at  this  stage,  output  {e{fn)}iqn+i)-  Since 

M  is  eventually  correct  on  input  /(O), /(I), . . .  ,/(n),  this  procedure  is  eventually 

correct  on  input /i(9o)5/i(9i),  •••  ,/j(9n)- 

Similarly,  if  C  €  NV^^^  via  G,  we  can  construct  an  M  to  NV-identify  C.  □ 
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Lemma  3.3.5.  C  e  EXcons  Ex£^^. 

Proof.  Suppose  M  EXcons-identifies  M,  and  let  h  e  C.  At  stage  n  +  1  we  wish  to 
produce  an  index  for  h.  The  procedure  is  similar  to  the  one  given  in  the  previous 
lemma.  Now,  h  =  e{f)  for  some  /  €  C,  so  from  h{qo),h{qi), . . .  ,h{qn)  we  may 
effectively  produce  /(O), /(I), . . .  ,/(n).  We  then  apply  M  to  the  sequence  of  /- 
values  to  produce  a  guess  e„  for  an  index  of  /.  Using  some  fixed  index  of  e,  viewed 
as  a  functional  on  the  partial  recursive  functions,  we  then  produce  an  index  ei,  from 
e„.  Eventually,  the  e„'s  will  converge  to  an  index  e  for  /,  so  that  e  is  an  index  of  h. 
The  other  direction  is  shown  similarly.  □ 

Theorem  3.3.6.  f:x£^^  -  NV^^^ 

Proof.  By  way  of  contradiction,  suppose  that  EX^^^^  =  NV^^^.  Then  by  the 
previous  theorem,  £X^„^^  =  PEX^^,^,  so  that  by  the  two  lemmas  above, 


But  then  Ny=  EXcons)  a  contradiction.  ,   *  .»      j-  □ 

The  inclusions  among  the  various  inference  types  defined  in  this  chapter  can 
be  summarized  as  follows:       •  '  '  ;  ■  >  .   '   '  f  [  \ 

PEX^,!^i^  =  NV^^^  C  EX£^^. 

Now,  we  may  define  the  classes  EX,  EX",  EX*,  BC,  BC",  and  BC  relative  to 
RUC  in  the  usual  way.  Then,  using  the  procedure  in  Lemma  3.3.5,  it  is  easy  to  see 
that  the  inclusions  among  the  various  classes  relativize  also.  In  the  next  chapter,  we 
will  formulate  new  notions  of  inference  which  allow  us  to  infer  RUC. 


CHAPTER  4 

APPROXIMATE  INFERENCE  OF  RECURSIVE  FUNCTIONS 
4.1    Notions  of  Approximate  Inference 

We  are  now  in  a  position  to  define  notions  of  inference,  NVoo,  EXoo,  and  BCoo, 
which  are  in  a  sense  "continuous"  analogues  of  the  standard  notions  JVV,  EX,  and 
BC.  The  idea  for  these  new  notions  comes  from  the  following  situation.  Suppose  that 
/  :  [0, 1]  — >  [0, 1]  is  continuous,  and  we  are  allowed  to  ask  "What  is  /(x)?"  for  (only) 
countably  many  x.  We  may  then  ask,  in  this  manner,  for  /(O),  /(I),  /(1/2),  /(1/3), 
/(2/3),  and  so  on.  After  we  ask  for  the  n'**  value,  we  can  then  form  an  approximation 
/„  to  the  graph  of  /,  perhaps  by  splines,  or  by  a  polynomial  interpolation.  Since 
we  know  that  /  is  continuous,  we  know  that  the  sequence  of  continuous  functions 
{/n}  converges  uniformly  to  /.  Thus,  we  have  a  procedure  whereby  we  can  build 
reasonable  approximations  to  /  in  stages,  with  the  knowledge  that,  "in  the  limit",  we 
recover  /  itself.  With  this  procedure  in  mind,  we  define  the  approximate  inference 
classes. 

Definition  4.1.1.  A  class  C  of  recursive  rational  functions  is  next-value  approx- 
imable  {C  G  iVVoo)  if  there  is  a  G  :  [/q]'^'^  — >  Iq  such  that  for  every  /  €  C 

lim|/(9„)-G((/(go),/(gi),...,/(9„-i)))|  =  0.  . 

This  definition  compares  to  standard  iVV-inference,  as  applied  to  rational  func- 
tions, where  G  infers  /  only  if 

/(9n)  =  G((/(9o),/(gi),...,/(9„-i))) 
for  all  but  finitely  many  n. 

■•   .  '  <    -      y  '-- 

;     '  -  ■  ■      -  .  ■  ;  ■     >  ■  ■  M 
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Definition  4.1.2.  A  class  C  of  recursive  rational  functions  is  explanatorily  approx- 
imable  {C  €  EXoo)  if  there  is  a  recursive  G  :  [/q]*''^  — >  N  so  that  for  every  f  €  C, 
there  is  an  index  e  such  that 

"  •  $e  is  total, 

•  for  all  but  finitely  many  n,  G{{f{qo), ...  ,  /(^n)))  =  e,  and 

•  \imn-^oo\f{qn)-^e{qn)\=0.  *  '     ' T  '  ) 

Thus,  an  EXoo-inference  machine  behaves  like  an  EX-inference  machine,  except 
that  the  index  it  settles  on  must  only  approximate  the  input  /  in  the  NVoo  sense. 
The  definition  for  C  G  PEXqo  is  similar,  but  G  must  output  only  indices  for  total 
functions. 

Definition  4.1.3.  A  class  C  of  recursive  rational  functions  is  behaviorally  correctly 
approximable  (C  €  BCoo)  if  there  is  a  recursive  G  :  [Iq]'^'^  — >•  N  such  that  for  every 

fee, 

Jim  11/  -  $G«/(,o)./(9i)  /M»lloo  =  0> 

where  H^Ho^  =  sup^gj^{|$(z)|}  if  $  is  total,  and  \\^\\^  =  1  if  $  is  paxtial.  The 
definition  for  C  G  PBCoo  is  similar,  but  G  must  output  only  indices  for  total  functions. 

This  definition  compares  to  standard  BC-inference,  as  applied  to  rational  func- 
tions, where  G  infers  /  only  if 

/  =  ^G({/(,o)./(9l)  /(9n-l))) 

for  all  but  finitely  many  n.  We  recall  that  the  more  restrictive  notion  of  EX-inference 
requires  further  that  there  is  a  fixed  e  such  that  G{{f{qo),f{qi),. . .  =  e 

for  all  but  finitely  many  n.  The  less  restrictive  notion  of  BC-inference  allows  G  to 
infer  /  if,  for  all  but  finitely  many  n, 

f{x)  =  $G((/(,o),/(9i)  /(9„-i)))(^) 
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f{qk)    k  <n, 

otherwise. 


except  on  a  finite  set.  We  note  again  that  the  class  QREC  is  in  BC,  but  that  neither 
QREC  nor  QSET  are  in  BC  '      ^^  *    ;.;        _  ...     .  .. 

An  obvious  way  to  weaken  the  above  definition  is  to  require  only  that  a  BCoo 
machine  G  output  guesses  {/„}  so  that  the  /„'s  converge  pointwise  to  the  input 
function  /.  The  modified  inference  notion  is  then  equivalent  to  J3C,  for  if  /  €  QREC, 
we  approximate  /  at  stage  n  by  /„,  defined  by 

fniQk)  = 

Note  that  by  definition  of  NV  (resp.  EX,  BC),  we  have  NV  C  NVoo  (resp. 
EXcEXoo,  BCcBCoo). 

4.2    Relationships  among  the  Inference  Classes 

The  results  of  this  section  determine  the  lattice  of  inclusions  among  the  infer- 
ence classes  NV,  EX,  BC,  NVoo,  EX^o,  PEX^o,  BCoo,  and  PBCoo-  As  was  previously 
mentioned,  the  standard  inference  notions  NV,  EX,  and  BC  can  easily  be  translated 
into  inference  notions  about  the  recursive  rational  functions.  We  shall  avoid  this  for- 
malism, taking  for  example  the  assertion  "NV  C  NVoo"  to  mean  that  the  analogue  of 
NV  (in  the  framework  of  functions  on  /q)  is  contained  in  NVoo. 

It  is  easy  to  see  that  restricting  approximate-inference  machines  to  output  only 
indices  of  total  functions  results  in  strictly  less  powerful  inference  notions: 

Theorem  4.2.1.  PEXoo  C  EXoo  and  PEXoo  C  PBCoo- 

Proof.  PEXoo  C  EXoo  is  trivial.  To  see  that  PEXoo  C  PBCoo,  let  C  €  PEXoo  via  G, 
and  define  a  PB Coo-inference  machine  G  zs  follows:  at  stage  n,  G  outputs  an  index 

for  $G«/(,o)./(9i),-./(9n-i)))  patched  with  the  values  {(go,/(go)), •  •  •  , (9n-i,/(9n-i))}. 

Then 

n-^  11-^  ~  ^G«/(,o),/(«i)....,/(«n-l)»lloo  "  ^' 

so  that  /  is  PB  Coo-inferred.  □ 
This  proof  yields  the  following: 
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Corollary  4.2.2,  EXoo  C  BCoo- 

Theorem  4.2.3.  PBC^  c  BCoo  and  PBCoo  C  NV^. 

Proof.  PBCoo  C  BCoo  is  trivial.  To  see  that  PBCoo  C  NVoo,  let  C  €  PBCoo  via  G, 
and  let  f  6  C.  Then 

lim  -  $G((/(,o),/(9l)  /(9n-i)»(9n)|  =  0,        ?  ^ 

so  that  /  is  NVoo-inferred.  .  □ 

Interpolation  of  known  function  values  is  a  natural  inference  procedure  in  the 
setting  of  recursive  rational  functions,  and  so  we  have  -  . 

Theorem  4.2.4.  R[/C€  PBCoo. 

Proof.  Let  /  e  RUC.  At  stage  n,  having  received  {f{qo),  f{qi),  •  •  •  ,  /(9n-i)),  simply 
output  (an  index  of)  the  linear  interpolation  /„  of  these  n  values.  The  functions  {/„} 
are  uniformly  continuous  on  /q,  and  converge  pointwise  to  /,  therefore  fn  f 
uniformly.  □ 

By  the  above  three  theorems,  we  have 
Corollary  4.2.5.  RUC  e  BCoo,  RUC  e  NVoc- 

However,  the  interpolation  method  cannot  be  used  to  EXoo-infer  RUC,  since 
it  requires  changing  indices  infinitely  often.  In  fact,  there  is  no  algorithm  which  will 
EXoo-infer  RUC.  The  idea  of  the  proof  is  essentially  the  Scime  as  that  in  the  proof 
that  REC  ^  EX. 

Theorem  4.2.6.  R[/C^  EXoo. 

Proof.  Suppose  G  EXoo-infers  RUC.  We  construct  an  /  G  RUC  on  which  G  changes 
its  mind  infinitely  often,  for  a  contradiction.  Let  {a,}  be  an  increasing,  computable 
sequence  of  rationals  with  limit  1,  and  set  /(O)  =  /(I)  =  0.  We  use  G  to  define  /  on 
all  of  Iq  as  follows.  At  stage  n,  if  9„  <  a„,  do  nothing.  Otherwise,  9„  €  [ak,Ok+i)  for 
some  k  >n.  Extend  /  in  two  ways:  fi  will  be  /  extended  by  0,  and     is  /  extended 
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with  a  "hat"  of  height  in  [ak,ak+i).  Both  /i  and  /j  are  in  RUC,  and  differ  by 
more  than  ^  on  a  positive  interval.  Thus,  there  is  a  stage  greater  than  n  at  which  G 
outputs  different  indices  on  /i  and  /2.  Extend  /  by  whichever  of  /i,  forces  G  to 
output  a  different  index  than  G  output  on  /  at  stage  n  —  1.  □ 

As  a  corollary  to  Theorem  4.2.2  and  Theorem  4.2.6,  we  have  ' 
Corollary  4.2.7.  EXoo  c  BCoo. 

In  addition,  although  PEX  =  PBC,  we  have  shown 
Corollary  4.2.8.  PEX^  c  PBCoo. 

The  interpolation  procedure  used  above  also  does  not  suffice  to  infer  an  arbi- 
trary RC  function.  We  show  that  RC  is  not  BCoo-  or  iVVoo-inferable.  We  will  need  a 
way  for  an  inference  machine  G  to  use  its  own  guesses  as  inputs: 

Definition  4.2.9.  For  any  G,  a  =  (oo, . . .  ,am-i),  we  define  T^a  on  Iq  by  recursion 
as  follows 

^((ao,...  ,a„-i)),  if  n<m, 

Tcdqn)   =    {  G{{ao,...  ,a,n-uRnd{TG,<j{qm)),--- , 

Rnd{TG,„iqn-i))),  otherwise, 


where 


■  „      :  r, 


Rnd(x)   =    <{  ^  ^ 

^  '  ^  otherwise. 


Theorem  4.2.10.  RC^  iVVoo.  -  ,  -■ 

Proof.  By  way  of  contradiction,  we  suppose  that  RC  E  NVqo  via  G  and  show  that 
we  can  construct  a  G  to  NV-infer  QSET.  Let  0  <  r  <  1  be  any  computable  irrational, 
and  let  {r,}  be  any  increasing  computable  sequence  of  computable  irrationals  in  [0, 1] 
with  limit  r.  Note  that  if  A  €  QSET,  then  A  €  RC,  where  we  define  A  by 

A(q)    =    I  ^'         ^  and  A{qi)  >  1/2, 

\  0,  otherwise. 

Construct  G  as  follows.  On  input  {f{qo),...  ,/(gn-i))  (from  /  €  QREC),  output 
"^G,{/(9o),->/(9m-i))(^")  largest  m  such  that  we  can  compute       from  f\^.  Then 
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if  A  6  QSET,  then  A  €  RC,  so  by  hypothesis,  G  outputs  guesses  within  1/3,  say,  of 
A{qn)  for  n  greater  than  some  N.  Let  M  denote  the  least  natural  number  greater 
than  such  that  A\j^  can  be  computed  from  A\j^^.  Then  if  A  €  QSET,  for  stages 
n  >  M,  G  outputs  A{qn).  □ 

A  A 

Note  that  A  can  also  be  computed,  uniformly,  from  A,  by  taking  A{qi)  =  A{qk), 
where  k  is  the  least  such  that  qk  €  (r,-,r,+i).  Thus,  there  is  a  primitive  recursive 
function  (f)  such  that  if  e  is  an  index  for  A,  then  4>{e)  is  an  index  for  A.  We  will  make 
use  of  (f>  in  the  following  theorem. 

Theorem  4.2.11.  RC^BCoo- 

Proof.  As  above,  we  suppose  that  RC  €  BCoo  via  G,  and  show  that  we  can  construct 
a  (?  to  BC-infer  QSET.  Let  r,  {r,}  be  as  above,  and  for  /,  define  /  as  previously.  Now 
let  /  €  QREC.  We  construct  G  as  follows.  On  input  {f{qo),---  ,/(9n-i))»  output 
4>G{{f{qQ), ...  1  /(^m-i))),  for  the  largest  m  such  that  we  can  compute  from 
Thus,  if  A  €  QSET,  then  A  G  RC,  so  by  hypothesis,  G  outputs  indices  of  functions 
that  are  everywhere  within  1/3,  say,  of  A  for  n  greater  than  some  A'^.  Let  M  denote 
the  least  natural  number  greater  than  A'^  such  that  A\ff  can  be  computed  from  A\f^f. 
But  then,  G  outputs  indices  of  A  for  n  >  M.  □ 

The  notions  BCoo,  EXooi  and  NVoo  are  the  /Q-domain  generalizations  of  EC, 
EX,  and  NV,  as  we  show  in  the  next  few  theorems.  Note  that  NV  =  PEX  and  that 
PEX  C  PEXoo  by  definition,  whence  NV  C  PEXoo-  For  completeness,  however,  we 
give  the  following  direct  proof: 

Theorem  4.2.12.  iVVc  PEXoo. 

Proof.  Suppose  C  €  iVV  via  G,  and  /  €  C.  We  define  a  PEXoo-inference  machine  H 
for  C  as  follows.  At  stage  n,  output  an  index  for  the  function  h  such  that  h{qi)  =  f{qi) 
for  each  i  <  n  and  h{qm)  —  G{{h{qo),h{qi),. . .  ,h{qm-i)))  for  each  m  >  n.  Since  G 
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makes  only  finitely  many  incorrect  guesses,  the  sequence  of  indices  output  is  eventually 
constant  (and  is  in  fact  an  index  for  /).  □ 

Theorem  4.2.13.  BCC  BCoo. 

Proof.  Containment  is  clear.  To  show  that  it  is  strict,  we  show  that  RUC  ^  BC. 
We  suppose  that  RUC  e  BC,  and  then  show  that  QREC  €  BC,  a  contradiction. 
Let  RUC  G  BC  via  G;  we  will  define  a  recursive  machine  H  which  5C-infers  every 
recursive  function  on  rationals.  Let  /  be  a  fixed  recursive  function.  First  recall  that 
for  any  rational  function  /,  we  may  define  an  associated  continuous  rational  function 
/  e  RUC  hy  defining  /(O)  =  0  and  f{l/i)  =  f{qi-i)/i  for  i  >  0  and  defining  /  for  any 
other  rational  point  by  linear  interpolation.  Then  the  values  of  /(?,)  may  be  computed 
for  Qi  >  l/n  from  f{qo),f{qi),  ■  ■  ■  ,/(g„_i).  Let  s{n)  be  the  least  s  such  that  q,+i  <  ^, 
so  that  lim„^oo5(n)  =  oo.  Now  let  E{{f{qo),f{qi),...  ,/(g„_i)))  =  G{{f{qo),f{qi), 
...  ,f{qs(n))))-  Since  /  G  RUC,  we  have  by  assumption  that  e„  =  E{{f{qo),f{qi), 
...  is  an  index  of  /  for  all  suflBciently  large  n.  It  is  now  straightforward 

to  compute  from  e„  an  index  H{{f(qo),f{qi),...  ,/(9n-i)))  for  the  function  which 
recovers  /  from  /.  That  is,  if  e„  is  an  index  for  the  function  /„,  then  H{{f{qo),f{qi), 
. . .  ,/(g„_i)))  is  an  index  for  the  function  /„  such  that  /„(g,_i)  =  i/„(l/z).  Thus  H 
infers  the  arbitrary  recursive  function  /.  ^  □ 

In  the  previous  literature  (see  Odifreddi  [17],  Angluin  and  Smith  [1],  and  Case 
and  Smith  [6]),  it  has  been  shown  that  NV  C  EX  C  BC.  The  next  few  theorems 
show  that  the  inductive  inference  hierarchy  is  no  longer  linear  when  we  consider  the 
approximate  inference  classes,  since  NVoo  and  BCoo  are  incomparable. 
Theorem  4.2.14.  EX  </:  m^. 

Proof.  Blum  and  Blum  [4]  show  that  the  class  {/  :  <;^y(o)  =  /}  is  EX-inferable.  It  is 
easy  to  see  that  C  =  {/  :  1/ /(O)  G  N  and  $i//(o)  =  /}  is  a  member  of  (the  analogue 
of)  EX,  in  the  context  of  recursive  rational  functions.  We  show  that  C  ^  NVqo.  For  a 
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contradiction,  suppose  that  C  €  NVoo  via  G.  By  the  Theorem,  we  may  define  ^t(e) 
as  follows.  Let  $t(e)(0)  =  1/e.  If  G(($t(e)(0)))  <  1/2,  let  $t(e)(l)  =  1,  otherwise  let 
=  0.  Continue  this  process  by  recursion,  so  that,  for  each  n,  $((e)(n  +  l)  =  1,  if 
G{{^t(e){qo),^t(e){qi),---  ,^t(e){qn)))  <  1/2,  and  $t(e)(n  +  l)  =  0,  otherwise.  Observe 
that  ^>((e)  is  a  total  recursive  function  for  all  e.  By  the  Fixed-Point  Theorem,  there 
is  an  index  e  s.t.  ^t(e)  =  ^e-  Then  $e  €  C,  but 

|$e(9n)  -  ^((^eCgo),  •  •  •  ,  ^e(gn-l)»|  >  1/2 

for  all  n,  a  contradiction.  CD 

Corollary  4.2.15.  EX  ^  PBCoo,  QREC  i  NV«>,  PBCoo  C  BCoo,  EXoo  NVoo, 
and  BCoo  t  NVoo- 

Proof.  EX  <^  PBCoo,  since  PBCoo  C  NVoo.  QREC  ^  JVVoo,  since  otherwise  EX  C 
NVoo.  PBCoo  C  BCoo,  since  if  PBCoo  =  -BCoo,  then  BCoo  C  NVoo  by  Theorem  4.2.3, 
so  that  EX  C  NVoo,  contradicting  Theorem  4.2.14.  Finally,  since  EX  C  EXoo  and 
EX  C  BC  C  BCoo,  the  last  two  assertions  are  true.  □ 

Proposition  4.2.16.  BC^  EXoo. 

Proof.  The  proof  of  EX^  -  EX     0  in  Blum  and  Blum  [4]  also  shows  that  EX^  - 

EXoo  7^  0.  .  □ 

The  next  theorem,  along  with  the  previous  corollary,  shows  that  NVoo  and 
BCoo  are  incomparable. 
Theorem  4.2.17.  NVoo  <t  BCoo- 

Proof.  Let  C  =  {f  e  QREC  :  lim„^oo /(g2n)  =  1/2, /(92n)  <  1/2  /(92n+i)  = 
0,  and/(92n)  >  1/2  /(92n+i)  =  1}.  C  €  NVoo  by  the  following  algorithm:  at  stages 
2n,  simply  output  1/2,  and  at  stages  2n  +  1,  use  the  f{q2n)  to  predict  whether  the 
next  value  will  be  0  or  1.  We  show  that  if  C  €  BCoo  (via  G,  say),  then  QSET  €  BC, 
a  contradiction.  We  define  a  machine  H  which  BC-infers  QSET  as  follows.  For  a 
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recursive  set  of  rationals  A,  define  fA{q2k)  =  1/2  +  1/k  if  qk  €  A,  fA{q2k)  =  1/2  -  1/fc 
if  g*;  ^  A,  and  /4(92*;+i)  =  XA{k).  It  is  clear  that  /a^C  for  any  A,  so  that  by 
hypothesis  /a  will  be  BCoo-infered  by  G. 

Now,  if  A  €  QREC,  on  input  (x>i(9o),  X^(?i),  •  •  •  ,XA(gn-i)>,  we  may  compute 
(/v4(9o),/vi(9i),  •  •  •  ,/A(92n-i))  as  defined  above  and  then  compute 

Since  /a  €  C,  there  is  an  AT  such  that  for  all  n>  N,v/e  have  W/a  -  $e„  ||  <  1/2.  Thus 
iovn>  N  and  any  q,  fA{q)  =  1  if  and  only  if  ^*e„(9)  >  1/2.  Now  use  the  Theorem 
to  compute  from  the  program  e„  a  program  H{{xA{qo),XA{qi),  ■  ■  •  ,XA{qn-i)))  =  On 
so  that 

r  1,  if$e„(g2fc+i)>i/2,  ; 

^an{qk)         I  otherwise. 

Since  XA{qk)  =  fA{q2k+i),  it  follows  that  H  BC-identifies  the  arbitrary  recursive  set 
A.  □ 

Corollary  4.2.18.  PBCoo  C  NVoo,  QREC  ^  BCoo- 

We  have  shown  that  the  relationships  among  the  approximate  inference  notions 
are  not  always  analogous  to  the  relationships  among  the  notions  of  standard  inference. 
In  the  sequel,  we  explore  the  reasons  for  this. 

4.3    The  Extended  Inference  Hierarchy 

Figure  4.1  illustrates  the  inclusions  derived  in  the  previous  section  among  the 
various  inference  notions.  When  we  add  the  notions  iVVoo  and  BCoo  to  the  picture, 
the  inference  hierarchy  is  no  longer  linear,  and  as  mentioned  earlier,  analogues  of 
some  theorems  of  "standard"  inference  no  longer  hold.  We  offer  a  heuristic  argument 
why  this  is  to  be  expected. 
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Figure  4.1:  The  extended  inference  hierarchy. 

In  the  "standard"  inference  setting,  it  is  easily  shown  that  NV  C  JBC,  but 
that  NV  —  PBC.  The  proof  that  PBC  C  NV  carries  over  to  approximate  infer- 
ence. The  reverse  inclusion  does  not.  Consider  the  usual  proof  that  NV  C  PBC. 
Given  C  G  NV  (via  M,  say),  and  f  £  C,  ai  stage  n  we  output  a  program  M„ 
which  on  input  k  <  n  computes  M{f{qo),f{qi),...  ,/(gjt)),  on  input  k  =  n  computes 
M((/(0),/(l),...  ,/(n-l),M((/(0),...  ,/(n- 1))))),  and  so  on  by  recursion.  Since 
M  is  completely  accurate  in  its  guesses  from  some  stage  N  onward,  the  program  M„ 
computes  /  if  n  >  N,  and  so  C  G  PBC.  ■ 

Suppose  we  try  to  translate  this  proof  to  the  new  inference  setting,  as  follows: 
given  C  G  NVqo  via  G,  and  f  E  C,  use  G„  (defined  analogously  to  M„)  as  a  guess 
for  /  at  stage  n.  Unfortunately,  this  does  not  work.  Roughly  speaking,  if  at  stage 
n,  G's  guess  for  /(?„)  differs  from  /(?„)  by  £  >  0,  however  small,  then  we  expect 
that  this  error  will  not  only  propagate,  but  worsen  as  we  ask  Gn  to  compute  on 
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inputs  qk  for  large  k,  so  that  for  /:  »  n,  \Gniqk)  —  fi<lk)\  >  0.  In  particular,  we  set 
Gn{qn+i)  =  G((/(9o),  •  •  •  ,/(gn-i),  Gn(9n))),  but  sincc  Gn{qn)  ^  /(^n),  wc  don't  know 
that  Gniqn+i)  —  f{<ln+i)  <  £•  In  fact,  even  assuming  a  strong  continuity  property  for 

G,  such  as 

\\x-y\\<e^\\Gix)-Gm\<e 

> 

we  may  only  conclude  that  |G„(7n+i)  —  /(9n+i)l  <  2e. 

Of  course,  in  the  proof  that  NV  C  PBC,  we  might  expect  M„'s  error  to 
propagate  in  the  same  way.  The  difference  is  that  M  is  accurate  from  some  finite 
stage  onward,  so  that  from  this  stage  on,  there  is  no  error  to  propagate.  In  contrast, 
we  cannot  be  sure  that  G's  guesses  are  ever  completely  accurate,  only  that  they 
get  "better"  as  time  goes  on.  Thus,  every  G„  magnifies  the  error  |G(/(go),/(9i), 
. . .  —  /(n)|  when  input  gjt,  fc  »  n,  so  that  we  cannot  expect  that  /  is  the 

uniform  limit  of  the  sequence  {G„}. 

Recall  that  all  of  the  standard  inference  notions  have  the  following  "finitistic" 
component  which  is  lacking  in  the  notions  of  approximate  inference  given  thus  far. 
Using  NV  as  an  example,  if  /  is  inferred  via  M,  then  by  some  (finite)  stage  N, 
M  will  ever  after  predict  correctly  the  next  value  to  be  input  from  /  (although  M 
does  not  "know"  when  this  stage  N  occurs).  Thus,  unlike  the  approximate  inference 
methods  defined  up  to  now,  there  is  a  criterion  for  accuracy  which  is  met  at  some 
finite  stage.  The  next  chapter  gives  methods  for  approximate  inference  which  also 
have  this  feature. 


CHAPTER  5 

WEAKER  NOTIONS  OF  APPROXIMATE  INFERENCE  ' 

5.1    The  "Epsilon"  Inference  Criteria      |  ;     I  ;  ;  (  ' 

Recall  the  motivation  for  approximate  inference  from  the  previous  chapter. 
/  :  [0, 1]  — >  [0, 1]  is  continuous,  and  we  are  allowed  to  ask  for  a  sequence  of  values 
f{x),  say  /(O),  /(I),  /(1/2),  /(1/3),  /(2/3),  and  so  on.  As  before,  after  we  ask  for 
the  n***  value,  we  may  construct  an  approximation  /„  to  the  graph  of  /,  by  some  form 
of  continuous  interpolation.  At  some  point  in  this  procedure,  we  may  be  satisfied 
that  we  are  "close  enough"  for  our  particular  purposes,  and  so,  no  longer  wish  to 
continue  to  build  the  approximations  {/n},  settling  instead  on,  say,  g  =  fs  for  some 
fixed  A'^  in  all  later  computations.  If  we  have  in  mind  that  we  wish  to  be  within,  say, 
£  =  0.00001  of  f{x)  for  all  x,  we  can  keep  asking  for  new  values  of  /(x),  checking 
these  to  make  sure  that  \g{x)  —  f{x)\  <  e.  If  we  eventually  happen  upon  an  x  for 
which  this  does  not  hold,  we  may  then  update  our  interpolant  g.  However,  since  the 
/„'s  converge  uniformly  to  /,  we  will  only  have  to  make  finitely  many  such  updates. 
Thus,  analogously  to  the  standard  inference  notions,  this  procedure  allows  us  to  meet 
our  inference  criterion  by  some  finite  stage,  although  we  cannot  in  general  determine 
when  this  stage  occurs. 

We  now  modify  the  notions  of  approximate  inference  accordingly  to  formalize 
this  idea.  The  BCoo  inference  criterion  is  essentially  one  of  uniform  convergence. 
Thus,  a  natural  weakening  of  this  criterion  is  to  require  convergence  of  the  guesses  only 
to  within  e  for  a  fixed  £  >  0.  We  may  similarly  weaken  other  approximate  inference 
criteria.  In  the  sequel,  we  take  e  to  be  rational,  for  purposes  of  computability. 
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Definition  5.1.1.  Let  £  >  0.  A  class  C  of  recursive  rational  functions  is  next-value 
approximable  to  within  £  (C  6  NVe)  if  there  is  a  G  :  [Iq]^'^  — >■  /q  such  that  for  every 

limsup|/(g„)-G((/(go),...  ,/(9n-i)))|  <£. 

n 

Definition  5.1.2.  Let  £  >  0.  A  class  C  of  recursive  rational  functions  is  explanato- 
rily approximable  to  within  e  {C  G  EXc)  if  there  is  a  G  :  [Iq]*^'^  — >  N  such  that  for 
every  f  G  C,  there  is  an  index  e  such  that 

•  $e  is  total, 

•  for  all  but  finitely  many  n,  G{{f{qo), . . .  ,/(gn-i)))  =  e,  and 

limsup|/(9„)  -  $e(gn)|  <  £. 

n 

The  definition  for  C  G  PEXj  is  similar,  but  G  must  output  only  indices  for  total 
functions. 

Definition  5.1.3.  A  class  C  of  recursive  rational  functions  is  behaviorally  correctly 
approximable  to  within  £  (C  G  BCc)  if  there  is  a  recursive  G  :  [Iq]^'^  — >■  N  such  that 
for  every  f  £  C,  ' 

limsup  11/  -  $G((/(,o)  /(9n-l)»Iloo  <^ 

n  ' 

So,  G  BCe-infers  C  if  for  each  /  €  C,  3A^  Vn  >  iV  G{{f{qo)J{qi), . . .  ,/(9„))  is  an 
index  of  a  recursive  function  whose  values  are  strictly  within  e  if  /'s  values.  The 
definition  for  C  G  PBCg  is  similar,  but  G  must  output  only  indices  for  total  functions. 

Each  of  the  above  criteria  yields  a  hierarchy  of  inference  classes  parameterized 
by  £  >  0.  Clearly,  for  £  >  1/2,  the  cleiss  le  contains  QREC  for  every  inference  notion 
I  G  {NV,EX,PEX,BC,PBC},  so  the  hierarchies  collapse  above  £  =  1/2.  We  will 
show  that  the  hierarchies  do  not  collapse  below  £  =  1/2,  and  are  in  fact  strictly 
monotone.  In  the  sequel,  S,e  range  over  rational  numbers  in  [0,1].  Let  6f  be  the 
function  obtained  from  /  by  pointwise  multiplication  by  5,  and  for  a  class  of  functions 
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C,  let  eC  denote  the  class  {Sf\f  e  C,0  <  5  <  e}.  It  is  easy  to  see  that  for  any  £  >  0,  if 
0<5<2€,  then  5QREC  is  an  element  of  each  I,  for  X  €  {NV,  EX,  PEX,  BC,  PBC}. 

We  begin  by  showing  that  these  new  inference  notions  are  strictly  weaker  than 
the  notions  of  approximate  inference,  in  the  sense  that  the  new  notions  infer  more 
classes  of  functions. 

Theorem  5.1.4.  For  any  £  >  0,  NVoo  Q  NV^.       •  '  ^- 
Proof.  Note  that  £QREC  G  NV,.  We  show  that  £QREC  ^  NVoo:  for  any  G  (which 
potentially  NVc^-infers  £QREC),  we  find  an  /  in  eQREC  not  inferred  by  G.  Let  * 
be  defined  by 

il>(n\=!'^  ifG((^(go),^(9i),...,'^(9„-i)»<!, 
"^^"^     \  0,  otherwise. 

Since  ^'  is  recursive,  and  <  £,  *  is  an  element  of  eQREC.  But 

lim sup  |*(g„)  -  G((^(9o),  ^(gi), . . .  ,  *(gn-i)))|  >  f , 
so  eQREC  is  not  NVoo-inferred  by  G.         .  '  □ 

Theorem  5.1.5.  For  £  >  0,  EXoo  C  EX,. 

Proof.  Note  that  eQREC  e  EX,.  We  show  that  eQREC  ^  EXoo:  Suppose  that 
eQREC  e  EXoo  ■  Then  £QSET  G  EXoo  (via  G,  say),  since  QSET  C  QREC.  But 
then  if  /  €  QSET,  feed  ef  to  G.  Let  e„  be  the  index  output  by  G  at  stage  n,  i.e. 
e„  =  G{{ef{qo),...  ,£/(g„_i))).  Then,  since  G  EXoo-infers  ef,  there  is  an  index  e 
and  a  stage  N  so  that  for  each  n  >  N,      =  e,  and 

lim|$e(9fc)-e/(git)|<|. 

We  use  this  to  EXoo-infer  QSET  as  follows.  For  each  n  we  output  an  index  of  the 
function  ^„  defined  by 

r  1,  if$e„(a:);>f, 

^„(X)=        0,  if$e„(x)i<f, 

[  undefined,  otherwise. 
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Since  e„  =  e  after  stage  N,  with  \^e{qk)  -  <  f  for  sufficiently  large  A;,  then 

because  /  €  QSET,  we  have  |^n(9*;)  -  /(9fc)|  =  0  for  such  k.  But  this  procedure 
is  uniform  in  /,  so  it  defines  a  EXoo-inference  machine  for  QSET,  a  contradiction. 
Hence  eQREC  €  EX,  -  EX^.  □ 

Theorem  5.1.6.  For  e  >  0,  BC^o  Q  BC,. 

Proof.  Note  that  e<QREC  e  BC,.  We  show  that  eQREC  ^  BCoo:  Suppose  that 
eQREC  e  BCoo  ■  Then  eQSET  G  BC^o  (via  G,  say),  since  QSET  C  QREC.  But 
then  if  /  e  QSET,  feed  ef  to  G.  Let  e„  be  the  index  output  by  G  at  stage  n,  i.e. 
en  =  G{{e  f{qo), ...  ,e /(gn-i))),  then  since  G  BCoo-infers  ef,  we  have 

lim||$e„-£/||oo=0, 

n 

SO  there  is  a  stage  A''  so  that  for  each  n  >  N,  ,  ,  ^. 

Il^e„-£/|loo  <f 

We  use  this  to  BCoo-infer  QSET  as  follows.  For  each  n  we  output  an  index  of  the 
function  ^„  defined  by  <  -    .    -  i^.  ■■ 

(    1,  if$e„(x)i>f, 

i'„(x)  =  <^  0,         if$e„(x);<f,   - 

y  undefined,  otherwise. 

Since  the  ||$e„  -  e/IL  <  f  after  stage  N,if  f  e  QSET,  then  ^„  =  /  after  stage  N. 
But  this  procedure  is  uniform  in  /,  so  it  defines  a  B Coo-inference  machine  for  QSET, 
a  contradiction.  Hence  eQREC  €  BCe  -  BCoo.  '  □ 

The  preceding  two  proofs,  mutadis  mutandis,  yield  analogous  results  for  PEXqo 

and  PB Coo: 

Corollary  5.1.7.  For  e  >  0,  PEXoo  C  PEX^. 
Corollary  5.1.8.  For  e  >  0,  PBCoo  c  PBC^. 
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Above,  we  exhibited,  for  each  e,  a  class  Cg  such  that  Cj  €  EXe  —  EXoo-  We 
can  further  show  that  there  is  a  class  C  with  C  €  OgEXc  —  EX^o,  and  that  C  G  EX^ 
uniformly  in  e.  This  class  C  is  just  RUC. 

Theorem  5.1.9.  RUC  £  EX,  for  all  e. 

Proof.  The  machine  which  accomplishes  this  is  a  simple  variant  of  the  linear 
interpolation  procedure  used  previously.  Fix  /  G  QREC,  let  L  be,  initially,  the 
zero  function,  and  let  Z/„  denote  the  linear  interpolation  of  {/{qo), . .  •  ,  /(^n-i))-  We 
construct  as  follows:  at  stage  n,  if  L{qn-i)  is  not  within  e  of  /(g„_i),  set  L  =  L„. 
Output  L. 

Now,  if  /  G  RUC,  eventually,  we  will  reach  a  stage  after  which  all  of  the 
linear  interpolants  L„  are  everywhere  within  e  of  /.  G,  will  stabilize  on  the  first  such 
interpolant.  .  □ 

In  the  case  of  JVVoo  it  is  straightforward  to  show  that  there  is  a  C  such  that 
C  €  n^iVVe  —  NVoo,  although  not  necessarily  uniformly.  For  each  n  >  0  and  x,  let 
Sn{x)  =  -^^x  +  ^.  Then  for  fixed  n,  for  any  /  €  QREC,  and  all  x  G  Iq,  we  may 
define  5'/,„(a:)  =  Sn{f{x)).  Now,  enumerate  QSET  arbitrarily  as  {/i,/2, . . .  }.  In  the 
next  two  theorems,  let  C  —  {«?/;,,},  and  let  v{n)  be  an  effective  enumeration  of  N,  in 
which  each  natural  number  appears  infinitely  often. 

Theorem  5.1.10.  NVoo  ^  aiVVe. 

Proof.  We  show  that  C  G  NVi/t  for  any  >  1,  but  that  C  ^  NVoo-  Fix  >  1,  and 
let  /,•  =  5/,,.-.  Then  C  G  NVi/*  via  Gk,  defined  as  follows  (input  /  G  QREC): 

Stage  n: 

Stfp  1:  Find  /  >  0  such  that  /(^n-i)  G  [^)        (^^  ^  exists —  i.e  is 

0  or  1 —  output  1,  and  goto  stage  n  +  1). 

Step  2:  U  I  <  k,  output  fi{qn),  otherwise,  output  ^. 
End  of  Construction.  .  •  - 


Clearly,  if  /  €  C,  then  Gk  NVi/t-infers  /.  Note  that  the  construction  of  Gk 
is  not  uniform  in  fc,  since  no  enumeration  of  QREC  is  effective.  Intuitively,  this  is 
why  C  is  not  NVoo-inferable.  We  suppose  C  is  NVoo-inferable,  via  G,  say  and  show 
that  we  can  then  NV-infer  QSET,  for  a  contradiction.  Construct  M  as  follows  (input 
/  e  QREC,  and  let  e  =  0):  -  >  -  , 

Stage  n: 

Step  1:  Compute  u„  =  G(5'/,^(e)|„). 

Step  2:  If  u„  is  closer  to        than  to  -^p^  output  1,  else  output  0. 

Step  3:  If  , 

\G{Sj,^(e)\r,.l)-  Sj,u(e){qn-l)\>-;^j;(^,  . 

increment  e. 

End  of  Construction. 

Now  if  A  e  QSET,  then  by  the  construction,  since  G  infers  C,  M  infers  A. 
Given  /  =  €  Q5ET,  let  /  =  /.•  =  5/,.-.  Then  /  €  C,  and  /(n)  =  ^  or  i  +  j^TT  for 
each  n.  Let  A'^  be  large  enough  so  that 

|G((/(9o),...,/(9n-i)))-/(9n)l<2^ 

for  all  n  >  N.  There  are  two  cases  to  consider.  - ; 

Case  1:  There  is  a  stage  m  >  N  where  i/{e)  =  i.  Then  for  every  n>  m 

and  S'/,„(e)(9n-i)  =  fiQn-i),  so  that  the  difference  computed  in  Step  3  is  less  than 
jtVt,  which  means  that  z/(e)  remains  equal  to  i  thereafter.   Thus,  for  all  n  >  m, 
~/(9n)|  <  2^T2>  so  ^hat  by  Step  2  of  the  construction,  since  /  G  QSET,  M{{f{qo), 
...  ,  =  /(g„)  for  all  n  >  m. 
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Case  2:  There  is  no  stage  m>  N  where  w{e)  =  i.  Since  i/(e)  =  i  for  infinitely 
many  e,  this  means  that  e  remains  fixed  from  some  stage  m  onwards.  But  this  implies, 
by  Step  3,  that  G  is  successfully  inferring  Sj^„(e)i(ln)  for  all  n  >  m.  It  follows,  as  in 
the  previous  case,  that  M  infers  /.  Note  that  in  this  case,  G  is  inferring  S /,^(e)  even 
though  <S'/,t/(e)  niay  not  be  in  C.  □ 

We  remark  that  the  above  proof  technique  is  independent  of  the  enumeration  of 
QREC  chosen,  and  therefore  does  not  work  to  show  that  BCoo  S  HcBCg.  To  see  this, 
enumerate  QSET  as  follows:  let  fi  =  0,  if  i  is  the  index  of  a  total  recursive  function 
which  is  not  identically  zero,  otherwise  let  fi  =  Xx.O.  Then  the  class  C  =  {5/;,,}  is 
EX-inferable  (and  therefore  B Coo-inferable)  by  the  machine  which  at  stage  n  outputs 

•  an  index  of  Ax.O  if  input  the  constant  sequence  (j^?  •  •  •  ,  length  n, 

•  the  index  e  if  input  any  other  sequence  (cq,  . . .  ,  On-i)  such  that  a,-  €  [^,  ^  + 
2^)  for  all  i, 

•  0,  otherwise. 

We  do  not  know  of  a  proof  for  BCoo  S  HBCe,  but  we  conjecture  that  the 

statement  is  true. 

5.2    Monotonicity  of  the  "Epsilon"  Hierarchies 

By  adapting  the  proofs  of  the  previous  section,  it  is  easy  to  show  that  the 

"epsilon"  hierarchies  do  not  collapse  for  e  <  1/2: 

Theorem  5.2.1.  For  any  0  <  £  <  1/2,  2€QREC  i  NV^.  '  ' 

Proof.  We  show  that  for  any  G  (which  potentially  NVj-infers  2eQREC),  we  can 
find  an  /  in  2eQREC  not  inferred  by  G.  Let  *  be  defined  by 

.(2e,  ifG((*(9o),^(9i),...,^(9n-i)))<£, 
\  0,  otherwise. 


Since     is  recursive,  and  W^^W^  <  2e,     is  an  element  of  2£QREC.  But 

lim  sup  ^(g„)  -  ^((^(^o),  *(gi), .  •  •  ,  *(gn-i)))  >  e, 

so  2eQREC  is  not  NVe-inferred  by  G.  ••  •  : ,     ]   <  ^  •  ff'  f   w  □ 

Thus,  the  {NVj}o<£<i/2  hierarchy  does  not  collapse: 
Corollary  5.2.2.  If  0  <  ei  <  £2  <  1/2,  then  NV,,  C  NV^,. 

Proof.  Any  NVsi -inference  machine  is  by  definition  an  NVe^ -inference  machine,  so 
NVji  C  NVj2.  This  containment  is  strict,  however,  by  the  preceding  theorem,  since 
2£iQREC  G  NVej,  via  G  which  simply  outputs  £1  on  any  input.  □ 

Also,  since  NVj  contains  both  PEXg  and  PBCg,  we  have:  •  . 

Corollary  5.2.3.  For  e  <  1/2,  2£QREC  ^  PBC,  and  2eQREC  ^  PEX^. 
Corollary  5.2.4.  If  0  <  ei  <  £2  <  1/2,  then  PBC„  C  PBC,^,  and  P£X,,  C 
PEX,,.  ■  ■ 

Theorem  5.2.5.  For  0  <  e  <  1/2,  2eQREC  i  EX,. 

Proof.  We  suppose  2£QREC  e  EX,,  for  a  contradiction.  Then  2£QSET  €  EX,  (via 
G,  say),  since  QSET  C  QREC.  But  then  if  /  €  QSET,  feed  2£/  to  G.  Let  e„  be  the 
index  output  by  G  at  stage  n,  i.e.  e„  =  G((2£/(go),  •  •  •  )2£/(gn-i)))-  Then,  since  G 
EXj-infers  2£/,  there  is  an  index  e  and  a  stage  N  so  that  for  each  n  >  N,  e„  =  e, 
and 

\\m\^e{qk)-2£f{qk)\<£. 

k 

We  use  this  to  EX-infer  QSET  as  follows.  For  each  n  we  output  an  index  of  the 
function       defined  by 

(  1,  if  $e„(a:)  ;>  £,  ' 

^„(x)=  I  0,  if  $e„(a:);<£, 

y  undefined,  otherwise. 

Since  e„  =  e  after  stage  N,  and  |$e(9Jt)  —  2£f{qk)\  <  £  for  sufficiently  large  k,  then 
l^n(<7fc)  —  fiQk)\  =  0  for  such  k.  But  this  procedure  is  uniform  in  /,  so  it  defines  a 
EX-inference  machine  for  QSET,  a  contradiction.  Hence  2£QREC  ^  EXg.  □ 
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Corollary  5.2.6.  If  0  <  ei  <  £2  <  1/2,  then  EX^,  C  EX,^. 

Proof.  Any  EXji -inference  machine  is  by  definition  an  EX^j -inference  machine, 
so  EXei  C  EXj2.  This  containment  is  strict,  however,  by  the  preceding  corollary, 
since  2£iQREC  €  EXj^,  via  G  which  simply  outputs  an  index  for  the  constant 
function/(g)  =  £1  on  any  input.  ■  □ 

In  the  following,  if  \g  —  f\  <  e,  we  will  call  g  an  e-variant  of  /. 
Theorem  5.2.7.  For  0  <  e  <  1/2,  2eQREC  ^  BC,. 

Proof.  We  suppose  2£QKEC  €  BC^,  for  a  contradiction.  Then  2£QSET  6  BCe  (via 
G,  say),  since  QSET  C  QREC.  But  then  if  /  €  QSET,  feed  2£/  to  G.  Let  e„  be  the 
index  output  by  G  at  stage  n,  i.e.  e„  =  G({2ef{qo),...  ,2ef{qn-i))).  Then,  since  G 

BCe-infers  2£/,  we  have 

<  ,  liin||$e„-2£/|U  =  0,  :■  .;; 

so  there  is  a  stage     so  that  for  each  n  >  A'^,        "  "*  ' 

ll^e„-2£/|U<£. 

Thus,  for  each  stage  n  >  N,  G  produces  an  index  of  an  e-variant  of  /.  We  use  this 
fact  to  BC-infer  QSET  as  follows.  For  each  n  we  output  an  index  of  the  function  ^„ 
defined  by 

C  1,  if$e„(x)i>£, 

^n{x)=l    0,  if$e„(x);<£, 

y  undefined,  otherwise. 
Since  the  ||$e„  —  2£ /||^  <  e  after  stage  A'^,  then  ^„  =  /  after  stage  N.  But  this  proce- 
dure is  uniform  in  /,  so  it  defines  a  BC-inference  machine  for  QSET,  a  contradiction. 
Hence  2EQREC  ^  BC,.  □ 

Corollary  5.2.8.  If  0  <  £1  <  £2  <  1/2,  then  BQ,  C  BQ,. 
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Proof.  Any  BC^, -inference  machine  is  by  definition  an  BCej-inference  machine,  so 
BCei  ^  BCc2-  This  containment  is  strict,  however,  by  the  preceding  corollary,  since 
2eiQREC  €  -BCej,  via  G  which  simply  outputs  an  index  for  the  constant  function 
f{q)  =  £i  on  any  input.  □ 

5.3    Relationships  among  the  Hierarchies 

Note  that  Theorems  4.2.14  and  4.2.17  are  easily  modified  to  yield  the  following: 
Corollary  5.3.1.  (to  4.2.14)  For  any  0  <  e  <  1/2,  EX<^  NV^. 
Corollary  5.3.2.  (to  4.2.17)  For  any  0  <  e  <  1/2,  NVoo  <t  BC,. 

Most  of  the  assertions  below  axe  just  corollaries  to  the  above,  and  to  the 
theorems  of  the  previous  section.  For  A,  B,  families  of  subsets  of  REC,  we  use  the 
notation  A  L  B  to  mean  A     B  and  B  ^  A. 

Corollary  5.3.3.  For  all  0  <  (J, £<  1/2,  NVs  1  BQ. 

Proof.  NV,  ^  BC5:  NVoo  C  NV„  but  by  5.3.2,  NVoo  t  BCs.  BCs  </L  NV,: 
EXcBCc  BCs,  but  by  5.3.1,  EX  <^  NV^.  □ 

Corollary  5.3.4.  For  all  0  <  5,e  <  1/2,  NV,  1  EXs. 

Proof.  NV,  <^  EXs:  NV^  C  NV„  but  by  5.3.2,  since  EXs  C  BCs,  NV^o  t  EXs. 
EXs  t  ^K:  EX  C  EXs,  but  by  5.3.1,  EX  (Ji  NVe.  □ 

Corollary  5.3.5.  If  0  <  e  <  1/2,  PBC^  C  BC^. 

Proof.  Containment  is  immediate.  It  must  be  strict,  since  otherwise  EX  C  NVg,  a 
contradiction.  □ 

Corollary  5.3.6.  If  0  <£<  1/2,  PBCe  c  Wg.  •         -  - 

Proof.  Containment  is  immediate.  It  must  be  strict,  since  otherwise  NVoo  C  BCg, 

a  contradiction.  ,  □ 
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Unlike  the  case  of  PEXqo  and  PBCoo  in  the  previous  chapter,  in  the  framework 
of  weak  approximate  inference,  we  can  easily  modify  the  proof  of  PEX  =  PBC  to  yield 

its  analogue:  -  ,        •      .    .  ,       ;  j' 

.  i  ^  i.  ■"   ^.  *  .       ■  ' 

Lemma  5.3.7.  If  0  <  £  <  1/2,  PEX,  =  PBC,. 

Proof.  If  C  G  PEX,  via  G,  on  input  /,  at  stage  n  we  patch  the  index  G{{f{qo), 
■  •  •  5  f{Qn)))  with  the  known  values  {f{qo), ...  ,  f{qn))-  U  f  €  C,  then  this  procedure 
serves  to  BC^-infer  /.  In  the  other  direction,  if  C  €  PBC,  via  G,  on  input  /,  at 
stage  n  simply  output  the  least  index  e,-  =  G((/(go),  •  •  •  ?/(9t))  (for  i  <  n)  for  which 
l^iiQx)  —  /(<?r)|  <  £  for  each  a:  <  n  (if  no  such  i  exists,  just  output  e„).  □ 

Corollary  5.3.8.  If  0  <  e  <  1/2,  PEX,  C  EX,. 

Proof.  Containment  is  strict:  otherwise,  since  PEX,  =  PBC„  we  would  have  EX  C 
NVe,  a  contradiction.  □ 

Previously,  it  was  noted  that  the  proof  of  EX^  -  EX  /  0  in  Blum  and  Blum  [4] 
also  shows  that  EX^  —  EXoo  7^  0.  In  fact,  it  is  easily  modified  to  yield  EX^  —  EX,  ^  0 
for  0  <  £  <  1/2.  Thus,  since  EX^  C  BC,  the  following  proposition  holds: 

Proposition  5.3.9.  If  0  <£<  1/2,  EXe  c  BCe. 

Figure  5.1  illustrates  the  inclusions  derived  in  this  section. 


PBCg=  PEXg 
Figure  5.1:  The  "epsilon"  hierarchy. 


CHAPTER  6 
INFERENCE  FROM  ORACLES 


6.1    Inference  from  Generic  Oracles 


We  now  turn  to  approximate  inference  using  oracles  (see  chapter  2  for  notation 
and  definitions).  We  would  like  to  obtain  analogues  for  approximate  inference  of  the 
results  in  Fortnow  et  al.  [9]  characterizing  oracle  triviality  for  EX  and  BC,  namely 
that 


where  Q{A)  is  the  condition  that  either  A  is  recursive,  or  A  <t  K  and  A  =t  G  for 
some  generic  G. 

As  noted  previously,  the  crux  of  the  argument  is  to  show  that  for  generic 
G,  EX[G*]  =  EX[G]  and  that  BC[G*]  =  BC[G].  It  appears  however,  that  similar 
relationships  do  not  hold  in  the  cases  of  iVVoo  and  BCoo-  In  the  sequel,  we  introduce 
the  concept  of  a  modulus  of  inference,  and  show  that  for  classes  C  which  can  be 
inferred  by  machines  M'^  with  recursive  inference  moduli,  only  finitely  many  queries 
are  needed.  This,  in  turn,  leads  to  new  notions  of  approximate  inference. 

We  recall  the  following  definition  of  genericity  provided  by  Fortnow  et  al.  [9]: 

Definition  6.1.1.  G  e  2'^  is  generic  if  for  each  Si  Vl^  C  {0,1}*,  either 

•  i3a-<G)  (Vr^a)r  ^  VV',  or 

•  {3a-iG)aeW. 

In  the  sequel,  we  will  use  the  following  formulation  (see  Jockusch  [13]  for  other 

characterizations). 

Lemma  6.1.2.  G  is  generic  if  and  only  if  for  each  Ilj  class  V  C  2'*',  either 


EX[A]  =  EX^  g{A)  and  BC[A]  =  BC  ^  g{A), 
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•  {3a^G)  I{a)cV,ov 

Proof.  {=^):  If  P  is  a  IlJ  class,  then  V  =  [T]  for  some  recursive  tree  T  (note  that 
T  is  Si).  U  G  ^  V  there  is  nothing  to  prove.  Suppose  that  G  E  V.  Then  Vtr  X  G, 
a  ^  T.  Thus,  by  genericity  of  G,  there  is  a  cr  -<  (7  so  that  for  each  t  y  a,T  ^  T, 
whence  /(a)  C  V. 

(<=):  Suppose  W  is  Si,  and  \f(T  ^  G,  a  ^  W.  Now  W  €  H?,  so  [W]  is  a 
n?  class,  and  G  e  [W].  Thus,  there  is  a  a  X  G  with  /(a)  C  [W],  so  for  all  r  >:  <t, 
r  §f  M^.  □ 

It  was  shown  [9]  that  for  generic  G,  EX[G]  =  EX[G*],  and  BC[G]  =  BC[G*]. 
EXoo[G]  =  EXoo[G*]  is  essentially  a  corollary  of  the  first  result.  However,  BCoo[G\  = 
BCoo[G*]  does  not  seem  to  follow  from  any  simple  modification  of  the  proof  BC|G]  = 
BC[G*].  We  suspect  that  in  fact,  BCoo[G*]  C  BCoo[G],  and  also  that  NVoo[G*]  C 
NVoo[G].  However,  we  can  show  that,  at  least  for  NVe-  and  BCe-inference  from 
generic  oracles,  no  more  power  is  obtained  from  an  infinite  number  than  from  an 
arbitrary  finite  number  of  queries.  The  basic  idea  is  to  compute  from  all  oracles  in  an 
interval  /(<t),  where  cr  -<  G,  as  long  as  the  various  computations  at  any  given  stage 
are  all  "close"  to  each  other.  If  not  all  the  computations  axe  close,  then  we  ask  for  a 
longer  initial  segment  cr  -<  r  -<  G,  and  start  computing  from  oracles  in  /(r).  Since  G 
is  generic,  we  will  only  have  to  return  to  G  finitely  many  times  to  obteiin  these  initial 
segments. 

Definition  6.1.3.  For  r  e  2<'^,  let  Sr  denote  the  partial  recursive  function  with 
domain  |t|,  defined  by  St{x)  =  {t)x.  We  say  that  r  is  {M,{xo,...  , Xn)) -minimal  if 
M^''{{xo, ...  ,  Xn))  4-,  but  for  all  proper  initial  segments  a  of  r,  M^''{{xo, . . .  ,x„))  t- 
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Lemma  6.1.4.  For  any  categorical        compact  V  C  2*^,  and  sequence  (cq,  . . .  ,  a„), 

the  set  ' 

{M^{{ao,...,an))\BeV} 

is  finite. 

Proof.  For  any  B  eV,  since  M^((ao, . . .  ,  a„))  |,  there  is  a  -<  5  so  that  M  uses 
only  ctb  in  its  computation.  Thus,  for  all  A  E  I{<^b),  M^((ao, ...  ,an))  =  A/'*((ao, 
. . .  ,  a„)).  Now,  {/((7b)  I  5  €  is  an  open  cover  of  V,  so  it  contains  a  finite  subcover 
/(ctbJ,...  ,/(crBj.  Thus, 

{M^((ao,...,an))|5€l>}=  z'^.  /; 

{M^>((ao,...  ,a„)),...  ,M^*((ao,...  ,a„))} 
is  finite.  □ 

Note  that  if  the  relation  B        Is  computable,  then 
{M^((ao,...,a„))|5€l>} 

is  uniformly  computable  in  {uq,  . . .  ,  a„). 

Theorem  6.1.5.  If  G  is  generic,  then  NVi/k[G*]  =  NVi/k[G]. 

Proof.  Let  C  e  NVi/k[G]  via  M^.  We  construct  M  to  iVVi/fc[G*]-infer  C.  Let 

/  G  QREC,  and  initialize  ctq  to  0.  Then  M  works  as  follows;  . 

Stage  n:  '  ■ 

Stepl:  Output  a„  =  M<^"°'((/(go),...,/(9„-i))).  ' 

5fej?  ^;  For  each  t  y  an  such  that  r  is  (M,  (/(go),  •  •  •  ,/(gn-2)))-niinimal  com- 
pute ., 

^       fx  =  M^((/(go),...,/(9„-2))). 

Note  that  since  M  is  categorical,  by  the  above  lemma,  all  such  r  can  be 
found  effectively.  Also,  the  list  may  be  empty,  since  M  might  need  only 
a  proper  initial  segment  of  <t. 


50 


Step  3:  If  for  some  r  we  have  \vr  -  f{qn-\)\  >  1/^,  let  <t„+i  =  G|^,  otherwise, 

let  cr„+i  =  cr„.      ,  ;  .  I 

End  of  Construction. 

If  /  G  C,  we  claim  that  M  makes  only  finitely  many  queries  to  G.  Since 
/  is  iYVi/jt-inferred  by  M^,  there  is  an  so  that  for  each  n  >  ATy,  |M^((/(go), 
...,/(g„_i)))-/(g„)|<l/A;.  Let 

Then  is  a  11?  class,  and  G  G  "P,  so  there  is  a  tr  G  with  /(a)  C  V.  Thus,  the 
consequent  of  step  3  is  invoked  only  finitely  often,  for  otherwise,  there  is  stage  n  >  Nj 
such  that  cr„  >-  a,  and  a  r  (7„  such  that  \vr  -  f{qn-i)\  >  1/^-  But  then  I{a)  ^  V, 
a  contradiction.  Thus,  M  queries  G  only  finitely  often. 

It  remains  to  show  that  M  infers  /.  Let  N  >  Nj  he  so  large  that  the  conse- 
quent of  step  3  is  not  invoked  after  stage  N,  and  fix  n  >  A'^  (note  that  cr„  =  Cjv).  If 
kn  —  f{(}n)\  >  ^/k,  then  there  are  two  cases  to  consider.  If  some  t  ^  cr^  was  used 
to  compute  a„,  then  a„  =  M^((/(go),  •  •  •  ,/(9n-i))),  contradicting  that  n  >  Nj.  On 
the  other  hand,  if  some  t  y  was  used,  then  at  stage  n  +  1  we  have  Vj  =  flnj  so 
that  the  consequent  of  step  3  is  invoked  at  this  stage,  contradicting  that  n  >  N.  □ 

Definition  6.1.6.  Let  M  be  any  inference  machine  which  outputs  indices.  The 
amalgamation  procedure,  AM,  is  defined  from  M,  a,  {xq,  . . .  ,  x„_i)  as  follows: 

Case  1:  (M''((xo,...  ,a;„_i))i).  Output  M^((xo,...  ,a;„-i)). 

Case  2:  {M''{{xq,...  ,a;„_i))  t)-    For  all  (M,(a;o,...  , x„_i))-minimal  t  y  a, 
compute  Ct  =  ^^((xo, . . .  ,  x„_i)).  Output  e  defined  by 

g.  (  \  _  j  ^er{^)j   for  the  first  r  s.t.  ^er{^)  i> 
*e(a:j  -  I  ^^^^^^  ^      ^^^j^  ^ 

End  of  Procedure. 


Theorem  6.1.7.  If  G  is  generic,  then  BCi/k[G*]  =  BCi/k[G].  -  ^  ^'  ' 

Proof.  Let  C  €  BCi/k[G]  via  M^.  We  construct  M  to  BCi/fc[G*]-infer  C.  Let 
/  €  QREC,  and  initialize  ctq  to  0.  Then  M  works  as  follows: 

Stage  n: 

Step  1:  Output  e„,  some  uniformly  chosen  index  of 

AMK,(/(go),...,/(9n-i)». 

Step  2:  For  each  r  )^  cr,  and  each  m  with  |<7„|  <  m  <  n  such  that  r  is  (M,  (/(^o), 
•••  i/(9m)))-niinimal  compute 

e;"  =  M^((/(go),...,/(9m))), 

and  for  each  x  <  n,  run  ^^"(a;)  =  $e?'(9i)  for  "  steps. 

Step  3:  If  for  some  r,  m,  and  x  we  have  v'^{x)  J.,  with  |uj^(a;)  —  f{qx)\  ">  1/fc,  let 
cTn+i  =  G|„.  Otherwise,  let  <t„+i  =  <7„. 

End  of  Construction. 

As  in  6.1.5,  for  f  E  C,  M  makes  only  finitely  many  queries  to  G.  Since  /  is 
BCi/fc-inferred  by  M^,  there  is  an  Nj  so  that  for  each  n  >  Nj, 

\\^MO({fM  /(?„-0»  - /Hoc  <  V^- 

Let      denote  M^{{f{qo), . . .  ,/(9„-i))).  Let 

P  =  {5  I  Vn>A^/ Vm  s.t.  (iV/<m<n)  Vx 

K^{^)i=^m^{x)-f{x)\<l/k}. 

Then  "P  is  a  IIj  class,  and  G  E  V,  so  there  is  a  cr  X  G  with  /(cr)  C  V.  Thus,  the 
consequent  of  step  3  is  invoked  only  finitely  often,  for  otherwise,  there  is  a  stage 
n  >  Nf,  X  <  n  and  r  >-  (t„,  with  cr„  X  cr,  such  that  lurCa;)  —  f{qx)\  >  1/^-  But  this 
contradicts  that  /(a)  C  P.  Thus,  M  queries  G  only  finitely  often. 
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Finally,  we  show  that  M  infers  /.  Let  a  =  lim„<T„,  N  >  max{A^/,  \a\},  and 
fix  n  >  A^.  If  |l$e„  -  /lloo  >  1/^,  then  there  is  an  x  with  \<^>e„{qx)  -  >  l/^- 

Thus,  by  definition  of  AM,  there  is  a,  t  y  cr  and  an  5  such  that  ^ln{qx)  i,  but 
l^e"(9i:)  -  /(<?r)|  >  1/^-  Then  at  stage  n  =  max{n,s  +  1},  the  consequent  of  step  3 
is  invoked,  contradicting  that  n  >  N.  □ 

Definition  6.1.8.  Let  M^^  be  a  categorical  oracle  T.M.,  let  A  €  2'^,  and  let  C  C 
QREC.  e  :  I^'^  — >  oj  is  an  NVoo-modulus  for  the  pair  (M^,C)  if  C  is  NVoo-inferred 
by  M^,  and  for  each  f  e  C,  3N  yk>  N  "^n, 

n  >  e{{f{qo), . . .  ,  f{qk)))      |M^((/(9o),  •  •  •  ,  /(9n-i)))  -  /(g„)|  <  1  A- 

£  :  /^'^  — >  Iq  is  a  BCoo-modulus  for  the  pair  (M'^,C)  if  C  is  BCoo-inferred  by  M^, 
and  for  each  f  eC,3N  yk>N  Vn, 

«  >  £((/(9o), . . .  ,  f{qk)))  ^  II ^M>»((/(,o),... ./(?„-!)))  -  /lloo  <  1/^- 

Without  loss  of  generality,  we  may  assume  that  e  is  increasing. 

In  the  sequel,  for  a  fixed  /,  we  will  often  abuse  notation,  denoting  e{{f{qQ), 
■■•  J{qn)))hy  e{n). 

Theorem  6.1.9.  Let  G  be  generic,  and  let  C  €  NVoo[G]  via        If  M  has  a  recursive 

JVVoo-modulus,  then  C  €  NVoo[G*]. 

Proof.  Suppose  NVoo-infers  C  with  recursive  modulus  e.  We  construct  M  to 
NVoo[G*]-infer  C.  Let  /  €  QREC,  initialize  ctq  to  0,  and  denote  by  kn  the  largest 
k  <  n  such  that  n  >  £{k).  Note  that  the  sequence  {fc„}  is  recursive,  uniformly  in  /. 
Then  M  works  as  follows: 

Stage  n: 

Step  1:  Output  an  =  M<'"°'((/(go),  •  •  •  , /(g„-i)». 
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Step  2:  For  each  r  )^  (7„  such  that  r  is  (M,  (/(go),  •  •  •  ,/(9n-2)))-niinimal  com- 
pute 

^r  =  M^((/(go),...,/(gn-2))). 

^iep  3:  If  for  some  r  we  have  \vr  -  /(g„_i)|  >  1/A;„,  let  <r„+i  =  otherwise, 
let  cr„+i  =  (7„. 

End  of  Construction. 

A 

Suppose  /  €  C.  We  claim  that  M  makes  only  finitely  many  queries  to  G. 
Since  /  is  iVVoo-inferred  by  M*^,  there  is  an  Kj  such  that  for  each  k  >  Kj  and  each 
n, 

n  >  e{k)  ^  |M^((/(9o),  •  •  •  , /(gn-i)))  -  /(9n)|  <  1/A:. 

Let 

V,^{B\yn>e{k)\M^{{f{qo),...J{qn-i)))-fiqn)\<l/k}. 

Then  "Pfc  is  a  11°  class.  Since  V  =  nfc>K^/  effective  intersection  of  11°  clcisses, 

and  G  eV,  there  is  a  cr  -<  G  with  /(a)  CV. 

Thus,  the  consequent  of  step  3  is  invoked  only  finitely  often,  for  otherwise, 
there  is  a  >  Kf,  a  stage  n  >  e{k),  with  |<t„|  >  A;  so  that  a  -<  cr„,  and  a  r  >-  <t„, 
such  that  \vr{x)  —  f{qx)\  ^  1/^-  But  then  I{(t)  <f.  Vk,  a  contradiction,  since  k  >  Kj. 
Thus,  M  queries  G  only  finitely  often. 

It  remains  to  show  that  M  infers  /.  Let  k  >  Kj,  and  >  e{k)  be  so  large 
that  the  consequent  of  step  3  is  not  invoked  after  stage  A'^,  and  fix  n  >  iV  (note 
that  an  —  a^)-  If  |an  —  /|  >  1/^,  then  there  are  two  cases  to  consider.  If  some 
T  -<  aN  was  used  to  compute  a„,  then  a„  =  M'^{{f{qo), . . .  ,f{qn-i))),  contradicting 
that  n  >  e{k)  and  k  >  Kj.  On  the  other  hand,  if  some  t  y  cr^  was  used,  then 
a„  =  M''{{f{qo), . . .  ,/(g„-i))),  so  at  stage  n  +  1,  we  have  that  \vr  -  f{qn-i)\  >  l/k, 
so  that  the  consequent  of  step  3  is  invoked,  again  a  contradiction.  □ 
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Theorem  6.1.10.  Let  G  be  generic,  and  let  C  €  BCoo[G]  via  M^.  If  M°  infers  C 
with  recursive  BCoo-modulus,  tlien  C  €  BCoo[G*\. 

Proof.  Let  M'^  have  modulus  e.  We  construct  M  to  BCoo[G*]-infer  C.  Let  /  € 
QREC,  initialize  (Tq  to  0,  and  denote  by  kn  the  largest  k  <  n  such  that  n  >  £{k). 
Then  M  works  as  follows: 

Stage  n: 

^^ep  1:  Output  e„,  some  uniformly  chosen  index  of 

AM(a„,(/(9o),...,/(9n-i))). 

5^ep  2:  For  each  r     cr,  and  each  m  with  |(T„|  <  m  <  n  such  that  r  is  (M,  {f{qo), 
/(gm)))-minimal  compute 

e7  =  M^i{f{qo),...J{qm))), 

and  for  each  x  <  n,  run  =  $e?'(9i)  for  steps. 

5fep  5;  If  for  some  r,  m,  and  x  we  have  vl?'(x)  l,  with  —  /(9x)|  >  1/A;m, 

let  <7„+i  =  G|„.  Otherwise,  let  <7„+i  =  <7n. 

End  of  Construction. 

Suppose  f  E  C.  We  claim  that  M  makes  only  finitely  many  queries  to  G. 
Since  /  is  B  Coo-inferred  by  M'^,  there  is  aa  Kj  such  that  for  each  k  >  Kj  and  each 
n, 

n  >  e{k)  ^  ||$M°((/(,o)  /(<;„-0))  -  f\L  <  .  ,       ,  ..  , 

Let      denote  M^((/(9o),...  ,/(9n-i))).  Let  _  , 

Vk  =  {B  \Wn>e{k)ym  s.t.  {£{k)<m<n)'rfx 

^:<n{x)i=^\^:^{x)-f{x)\<i/k}. 

Then  Vk  is  a  class.  Since  P  =  CI^^k^  Vk  is  an  effective  intersection  of  IIJ  classes, 
and  G  eV,  there  is  a  a  X  G  with  /(a)  C  V. 
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Thus,  the  consequent  of  step  3  is  invoked  only  finitely  often,  for  otherwise, 
there  is  a  >  Kj,  a  stage  n  >  e(fc),  with  |cr„|  >  A;  so  that  the  consequent  of  step  3 
is  invoked  at  this  stage.  So,  there  is  an  x  <  n,  and  m  with  |<t„|  <  m  <  n,  such  that 
a  -<  an,  and  a  t  y  a^,  such  that  \v!^{x)  -  /(g^)!  >  But  then  I{a)  (fi  Vk^,  a 

contradiction,  since  km'>  k  >  Kf.  Thus,  M  queries  G  only  finitely  often. 

It  remains  to  show  that  M  infers  /.  Let  k  >  K/,  and  >  e{k)  be  so  large 
that  the  consequent  of  step  3  is  not  invoked  after  stage  N,  and  fix  n  >  A'^  (note 
that  (T„  =  aN).  If  ||$e„  -/IL  >  then  for  some  x,  |$e„(gx)  -  /(9x)|  >  l/k. 
There  are  two  cases  to  consider.  If  some  r  -<  was  used  to  compute  e„,  then 
e„  =  M^((/(go),  •  •  •  ,/(9n-i))),  contradicting  that  n  >  e{k)  and  A;  >  Kj.  On  the 
other  hand,  if  some  t  y  aN  was  used,  then  by  definition  of  AM,  ^e„{qx)  = 
for  some  m.  Now,  if  m  >  n,  then  we  have  l*Mr((/(,„),...  j(,„_,)))(gx)  -  /(9x)|  >  1/^, 
so  that  /(<t)  ^  Vk,  a  contradiction,  whence  m  <  n.  But  in  this  case,  since  n  —  1  > 
£{k),wehavethatkn-i  >  k.  Thus,  since  =  M'"((/(go),  •  •  •  >/(9n-i))),  at  stage  n  we 
have  —  >  1/kn-i,  so  that  the  consequent  of  step  3  is  invoked,  which 

is  again  a  contradiction,  since  n  >  N.  □ 

6.2    Inference  with  Recursive  Moduli 

Definition  6.2.1.  We  say  C  G  MNVoo[A]  if  there  is  a  categorical  O.T.M.  G^^  such 
that  C  e  NVoo[A]  via  G'^,  with  recursive  iWoo-modulus  e.  We  define  MB  Co©  [A] 
similarly.  We  say  C  €  MNVqo  if  C  €  MN Vqo [A]  for  some  recursive  A,  and  define  and 
MBCoo  similarly. 

We  show  in  the  sequel  that  the  classes  defined  above  lie  strictly  between  the 
corresponding  standard  and  approximate  inference  classes. 

Theorem  6.2.2.  iVyc  MiVVoo. 
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,  ^      ..       ,    :-.  ■  ■  •  r  .       ^  . 
Proof.  Note  that  there  is  a  single  recursive  function  which  works  as  an  iVVoo- 
modulus  for  all  iVV-inference  machines  M,  and  C  G  NV,  namely 

e((ao,ai, . . .  ,an-i))  =  n. 

Thus,  NV  C  MNVoo.  Now,  for  /  €  QREC,  define  /  by  f{x)  =  f{x)/{x  +  1).  Let 
C  =  {f  \  f  e  QREC}.  Then  C  e  MNV^o  via  M  which  outputs  0  on  any  input.  The 
modulus  function  is  given  by 

£((ao,ai,-  • .  ,an-i))  =  n. 

However,  it  is  easy  to  see  that  C  ^  NV.  Suppose  C  €  NV  via  M.  Then  we  can  use 
M  to  build  a  machine  M  as  follows: 

M((/(9o), . . .  ,/(gn-i)))  =  (n  +  1)  *  M((/(9o), . . . 
It  is  easy  to  see  that  M  NV-infers  QREC.  •  □ 

Theorem  6.2.3.  BCC  MBCoo. 
Proof.  As  with  NV-inference, 

£{{ao,ai,...  ,a„_i))  =  n. 

works  as  an  BCoo-modulus  for  all  BC-inference  machines  M,  and  C  €  BC.  Thus, 
BC  C  MB  Coo-  For  /  €  QREC,  define  /  by  linear  interpolation  of  the  following  points: 
/(1/n)  =  (1/n)  */(?„),  with  /(O)  =  0,  and  /(I)  =  1.  Let  C  =  {/  |  /  €  QREC}.  Then 
C  C  RUC,  so  C  G  BCoo  by  Theorem  4.2.4  (via  a  machine  which  at  stage  n  outputs 
the  linear  interpolation  of  the  n  inputs).  A  recursive  modulus  function  is  given  by 

£((ao,ai, . . .  ,an_i))  =  A;, 

where  k  is  greatest  such  that  1/k  €  {qo,  qi,...  ,  ^n-i}-  However,  it  is  easy  to  see  that 
C  ^  BC.  Suppose  C  e  BC  via  M.  We  use  M  to  build  a  machine  to  M  defined  as 
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follows:  on  input  {/{qo) 


,/(<?n-i))),  M  outputs  an  index  of  the  function  <^n(^) 


defined  by 


MQm)  =  m  *  ^M((/(,o),..../(9„-i)»(l/"')- 


Then  M  BC^infers  QREC. 


□ 


Theorem  6.2.4.  MiVVoo  c  NVoo.  :  ; 

Proof.  We  suppose  that  RUC  is  NVoo-inferred  by  M,  with  recursive  NVqo  modulus 
£,  for  a  contradiction.  The  proof  uses  a  variation  of  the  diagonalization  construction 
used  previously  in  Theorem  3.3.1:  use  M,  e  to  construct  /  in  RUC  for  which  the 
modulus  e  fails  infinitely  often.  We  construct  /  as  follows  (set  /(O)  =  /(I)  =  0,  and 
let  qo  =  0): 


Step  1:  If  q-n  <  (j„,  then  f{qn)  is  already  defined,  so  do  nothing. 

Step  2:  Otherwise,  let  u„  =  M{{f{qo), . . .  ,/(gn-i))),  and  let  k  be  greatest  such 
tha.t  n>  e{{f{qo),...  ,f{qk))). 

Step  3:  If  Vn  >  then  we  violate  e{{f{qo),...  ,fiqk)))  by  extending  /  by 

zero  to  Qn  (i.e.  set  f{q)  =  0  for  all  q  €  [9ru9n]).  On  the  other  hand,  if 
Vn  <  l/k,  we  violate  £{{f{qo)i---  ,f{qk)))  by  extending  /  with  a  "hat" 
of  height  |.  Thus,  we  define  f{q)  for  q  €  [9nj9n]  by  linear  interpolation 
of  the  three  points  (g„,0),  (^^^y^,  I),  and  (9„,0). 

Step  4:  Set  9„  -  -  maxj^o,---  ,9n}. 

End  of  Construction. 

Then  /  €  RUC,  but  by  the  construction,  e  fails  infinitely  often  on  /.  □ 

Theorem  6.2.5.  MBCoo  c  BCoo. 


Stage  n: 
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Proof.  We  suppose  that  RUC  is  B Coo-inferred  by  M,  with  JBCoo  modulus  e,  for  a 
contradiction.  As  above,  we  use  M,  e  to  construct  /  in  RUC  for  which  the  modulus  e 
fails  infinitely  often.  We  construct  /  as  follows  (set  /(O)  =  /(I)  =  0,  and  let  Qq  =  0): 

Stage  n: 

Step  1:  If  Qn  <  Qn  then  f{qn)  is  already  defined,  so  do  nothing. 
Step  2:  Otherwise,  g„  >       Let  /  be  defined  by 

■' 1^  0,  otherwise, 
and  let  k  be  greatest  such  that  n  >  e{{f{qo), ...  ,  f{qk)))- 

'  -  ■    y      \  ■; 

Step  3:  Dovetail  computations  of  {^m(/|  )(9n)}«>n  ^^^^^  ^  convergent 

computation  Un  =  $^/;|  ,(g„).  -f^-,-      ^.  „.v 

Step  4:  Let  r  =  max{g„,  g,„}.  If  u„  >  1/k,  then  we  violate  £{{f{qo), ...  ,  f{qk))) 
by  extending  /  by  zero  to  r  (i.e.  set  f{q)  =  0  for  all  q  €  [^n,'']).  On  the 
other  hand,  if  i;„  <  1/A;,  we  violate  e{{f{qo), . . .  ,f{qk)))  by  extending  / 
with  a  "hat"  of  height  |.  Thus,  we  define  f{q)  for  q  €  [qnif]  by  linecir 
interpolation  of  the  three  points  (9n,0),  (^-j^,  |),  and  (r,  0). 

5^ep  5;  Set  ^„  =  max{go,  •  •  •  >  J"}' 

End  of  Construction. 

Then  /  £  RUC,  but  by  the  construction,  s  fails  infinitely  often  on  /.  □ 

•c 

6.3    Inference  from  Sets  of  Oracles 

We  now  turn  to  a  different  question  concerning  the  use  of  oracles  in  inductive 
inference,  one  not  addressed  in  Fortnow  et  al.  [9].  Let  ^()  denote  the  usual  measure 
on  2'^.  We  could  then  define  the  following  inference  notions: 
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Definition  6.3.1.  A  class  C  of  recursive  rational  functions  is  next-value  approx- 

imable  almost  surely  [C  €  iVV^)  if 

fi{{A\CeNV^[A]})  =  L 

And  more  generally,  we  could  define  an  apparent  hierarchy  of  notions  NV°  for 
0  <  a  <  1  byC  €  NV^  if 

li{{A  I  C  €  NVo,[A]})  >a.  \  . 

We  shall  see  in  the  sequel  that  this  hierarchy  collapses,  i.e.  for  all  a  >  0,  NV^  =  JVVoo. 
In  what  follows,  we  will  show  that  the  analogous  hierarchies  collapse  for  the  other 
notions  of  standard  and  approximate  inference.  The  technique  we  employ  is  one 
similar  to  that  used  by  Sacks  [23]  to  show  that 

fx{{B  \A<tB})>0=^A=tO. 

i'.       Definition  6.3.2.  Let  A,Be  2"',  and  let  J  be  any  inference  notion.  Then  A  is 
■        X-trivial  if  X[A]  =  J.  I[A]  C  X[B]  is  denoted  A  <x  B.  The  X-cone  above  A  is  the 
set  of  oracles  {B  \  A  <j  B}. 

Theorem  6.3.3.  Let  C  c  REC.  If  n{{A  \  C  e  NV[A]})  >  0,  then  C  e  NV. 
Proof.  Let  S  =  {A  \  C  e  NV[A]},  and  suppose  fi{S)  >  0  (note  that  S  is  mea- 
surable). Let  Sn  —  {A  \  C  ^  NV[A]  via  ^^}.  Since  S  =  {J^^n,  there  is  an  e  with 
fj.{Se)  >  7a  for  some  positive  rational  a.  Now,  there  is  an  open  set  Q  containing 
<Se,  with  fi{Q  —  Se)  <  a.  Furthermore,  since  Q  is  open,  there  is  a  basis  element 
B  =  \Jl-i  I{crk)  contained  in  Q  with  fi{Q  —  B)  <  a. 

Now,  suppose  f  £  C.  Then  we  may  stratify  Se  according  to  how  soon  /  is 
inferred,  as  follows.  For  each  A'^  €      let  ,  .     j  .    j  -       .  .  v- 

Se,N  =  {BeSe\\/n>N  0f  ((/(O), . . .  , /(n  -  1)))  =  /(n)}. 
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Note  that  {Se,N}N=Q  is  nondecreasing,  with  Se  =  Un^^^.n-  Thus,  for  any  f  £  C, 
there  is  an  A^;  such  that  for  all  N  >  Nj,  n{Se  -  Se,N)  <  a-  Hence,  for  >  AT;, 
IJ,{Se,N  r\B)  >  4a,  and  fi{B  -  Se,N)  <  2a.  Figure  6.1  illustrates  this  situation. 


Figure  6.1:  The  relationship  between  Se,N  and  B  foi  N  >  Nj. 


We  now  give  the  construction  of  M  to  NV-infer  C  (let  f  E  C): 

Stage  n: 

Step  1:  Compute  vb  =  </>f  ((/(O), . . .  ,  /(n  -  1)))  for  all  oracles  BeB.  ^ 

Step  2:  Output  the  least  value  v  for  which  n{{B  eB  \  vb  =  u})  is  greatest. 
End  of  Construction. 

Since  is  a  finite  union  of  intervals,  only  a  finite  number  of  computations  are 
made  in  step  1  at  any  stage  n.  Not  all  of  the  <^f  necessarily  NV-infer  C,  but  every 
computation  in  step  1  terminates,  since  is  categorical.  Thus,  the  algorithm  is 
well-defined,  and  total  for  any  input  /.  Now,  if  /  6  C,  then  for  all  stages  n  >  Nj, 
we  have  <5e,iV;  C  {B  £  B  \  vb  =  /(j^)}?  so  that  n{{B  £  B  \  vb  =  /(«)})  >  4a.  Thus, 
since  for  each  n  >  iV/,  ii{B  —  Se,n)  <  2a,  M  outputs  f{n)  after  stage  Nj.  Q 

Definition  6.3.4.  The  majority  amalgamation  procedure,  MAM,  is  defined  as  fol- 
lows. From  parameters  /  =  {ei,  62, . . .  ,  ejt},  and  W  =  {u^i,  iwj, . . .  ,  lojt},  on  input  x, 
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dovetail  the  computations  <^e,(x),...  ,(l)e^{x),  until  ek^ek^,...  ,ek„,  axe  found  such 
that  each  (pek^i^)  converges  to  some  common  value  u,  and  Yl^=i'^ki  >  ^/^^^=i^j- 
Output  v;  if  no  such  common  value  exists,  then  MAM{I,  W){x)  t- 

We  give  the  analogous  result  for  BC  before  the  EX  proof,  since  the  BC  case 
is  easier. 

Theorem  6.3.5.  Let  C  c  REC.  If  n{{A  \  C  e  BC[A]})  >  0,  then  C  e  BC. 
Proof.  We  define  S,  Se,  {Se,N}N=Q^  Q  ^"^^  ^  analogously  to  the  sets  of  Theo- 
rem 6.3.3.  Let  S  =  {A\C  e  BC[A]},  with  n{S)  >  0.  For  each  n,  set  5„  =  {A  |  C  € 
BC[A]  via  0f }.  As  before,  there  is  an  e  with  /i(<5e)  >  7a  for  some  positive  rational  a, 
an  open  set  ^  containing  Se,  with  n{Q  —  Se)  <  a,  and  a  basis  element  B  =  Ufc=i  ^i^k) 
contained  in  Q  with  -  B)  <  a.  If  /  €  C,  we  stratify  Se  according  to  how  soon  / 
is  inferred,  as  follows.  For  each  N  E  u>,  let 

Se,N  =  {B  eSe\yn>N  4>f{{m,  ...,f{n-  1)))  is  an  index  of  /}, 

so  that  {Se,N}N=o  is  nondecreasing,  with  Se  =  Un^^c.n-  Thus,  for  any  f  £  C, 
there  is  an  Nj  such  that  for  all  A'^  >  Nj,  iJ,{Se  -  Se,N)  <  a.  Hence,  for  N  >  Nj, 
IJ,{Se,N  f^B)  >  4a,  and  iJ.{B  -  <Se,iv)  <  2a. 

We  construct  M  to  BC-infer  C  as  follows. 

Stage  n: 

Step  1:  Compute 

In  =  {eB,n  I  es,„  =  <^f  ((/(O), . . .  ,/(n  -  1))),  some  B  €  B}. 

Step  2:  Enumerate  the  elements  of  /„  in  increasing  order  ei,  ej, . . .  ,  e^.  For  each 
i  <  k,  compute 

Wi  =  fi{{BeB\4>f{{m,...Jin-l)))  =  ei}). 

Denote  the  set  of  all      by  W{In)-  ^, 

:        - .    .  .     ■    :  :    n  ,. 


Step  3:  Output  an  index  e„  (uniformly  chosen)  of  MAM(/„,  W(/„)). 

End  of  Construction. 

Analogously  to  Theorem  6.3.3,  at  every  stage  n,  the  computation  in  step  1 
terminates,  producing  a  finite  set  of  indices  /„.  If  /  G  C,  then  for  all  stages  n  >  Nj, 
we  have 

Se,N,  C  {B  e  B  \  (i)f  IS  an  index  of  /}, 

so  that  for  any  X,  '  ' •> 

^{{B  e  B  \  <f>e^Jx)  i=  fix)})  >  4a. 

Thus,  for  any  n  >  Nj,  MAM(/„,  W{Q)     f.  -  □ 

Theorem  6.3.6.  Let  C  C  REC.  If /x({A  |  C  e  EX[A]})  >  0,  then  C  €  EX. 
Proof.  We  define  <S,  Se,  {<5e,iv}?/=0)  ^  ^  analogously  to  the  sets  of  Theo- 
rem 6.3.3.  Let  <S  =  {A  1  C  6  BC[A]},  with  n{S)  >  0.  For  each  n,  set  5„  =  {A  |  C  € 
BC[A]  via  </)f }.  As  before,  there  is  an  e  with  /x(<Se)  >  7a  for  some  positive  rational  a, 
an  open  set  Q  containing  <Se,  with  —  ^e)  <  a,  and  a  basis  element  B  =  (Jfc=i  K^k) 
contained  in  Q  with  —  B)  <  a.  U  f  E  C,  we  stratify  «Se  according  to  how  soon  / 
is  inferred,  as  follows.  For  each  N  £  u,\et 

{BeSel  3eB  Vn>  iV  <^f  ((/(O), . . .  ,/(n  -  1)))  =  eg  and  ,^e«  ~  /}, 

so  that  {<Se,A^}N=o  is  nondecreasing,  with  Se  =  Uiv*^e.^-  Thus,  for  a,ny  f  E  C, 
there  is  an  N/  such  that  for  all  A'^  >  Nj,  fi{Se  —  S^n)  <  a-  Hence,  for  N  >  Nj, 
fj.{Se,N  r\  B)  >  4a,  and  so,  iJ,{B  —  S^n)  <  2a. 

Let  eB,n  denote  <^f  ((/(O), . . .  ,/(n  —  1))).  In  the  remaining  theorems  of  this 
section,  we  will  denote  the  set  of  indices  produced  by  oracles  B  E.  Aior  some  A  C2'^ 
at  stage  n  as  follows: 

Ind{A)  =  {eB,n\BeA}. 
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Also,  let  lo  =  0,  and  let  /„  =  Ind{Xr,),  and  F„  =  Ind{J^n)-  We  construct  M  to 
EX-infer  C  as  follows  (let  n  >  0):  ; ' 

Stage  n: 

Step  1:  Set  'P„  =  I„_i,  and  ^„  =  {5  €  ^  |  eB,„  =  eB,„_i}  (note  that  ^„  is 
computable,  since     is  a  finite  union  of  intervals). 

Step  2:  If  /i(P„  n  :F„)  <  4a,  let  2„  =  :r„,  otherwise  let  2„  =  "Pn- 

5^ep  S";  Compute  W{In)  as  in  Theorem  6.3.5. 

Step  4:  Output  an  index  (uniformly  chosen)  of  MAM(/„,  ]y(/„)). 

End  of  Construction.  >-  .  ■ 

Analogously  to  Theorem  6.3.3,  at  every  stage  n,  the  computation  in  step  1 
terminates,  producing  a  finite  set  of  indices  /„.  If  /  €  C,  then  for  all  stages  n  >  Nj, 
we  have  Se,Nf  C  J^n,  so  that  ^(F„)  >  4a.  Since  n{B  —  Se,n)  <  2a  for  n  >  N/,  for 
these  n,  the  set  of  indices  of  /  has  relative  weight  greater  than  1/2  in  F„.  Thus,  for 
any  n  >  Nf,  MAM{Fr,,W{Fr,))  ~  /.  ' 

We  claim  that  for  n  >  Nj,  the  consequent  of  step  2  is  invoked  at  most  once, 
and  that  lim„  M((/(0), . . .  ,/(«)))  exists  and  is  an  index  of  /.  If  the  consequent  of 
step  2  is  invoked  after  stage  Nf,  let  n  be  the  stage  of  the  first  such  invocation.  Then 
at  this  stage,  we  set  Xfi  =  Tn-  Since  n  >  iV/,  for  all  n  >  n,  \i{Tn  fl  Tf^  >  4a,  so  that 
the  consequent  of  step  2  is  not  invoked  at  stage  n,  whence  Xn  =  Tn  for  all  n  >  n. 
But  then,  as  noted  previously,  MAM(/„,  W{In))  ~  /,  so  M  EX-infers  /. 

Suppose,  on  the  other  hand,  that  the  consequent  of  step  2  is  never  invoked  at 
any  stage  n  >  Nj.  Then  there  is  a  (computable)  set  of  oracles  I  with  n(X)  >  4a,  so 
that  for  all  n  >  Nj,  In  —  I.  But  then,  for  all  n  >  Nj,  M  outputs  a  canonical  index 
of  MAM{Ind{I),  W{Ind{I))).  We  claim  that  MAM{Ind{I),  W{Ind{I)))  computes  /. 
Fix  n  >  Nj.  By  hypothesis,  //(.F„nJ)  >  4a.  And  by  choice  of  n,  /x((5-5e,„)nl)  <  2a. 
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Thus,  the  set  of  indices  of  /  in  Ind{I)  has  relative  measure  greater  thcin  1/2,  so  that 

MAM{Ind{I),W{Ind{I)))  computes  f.  □ 

Definition  6.3.7.  Let  A  =  {ai,...  ,ak}  and  W  =  {wi,...  ,Wk}  be  finite  sets  of 
rationals  {W  is  called  a  weight  set  for  A,  and  u;,-  is  called  the  weight  of  a,).  Let  M  >  0. 
We  say  that  5  C  A  is  an  \ I M -majority  set  for  {A,W)  if  X)oi€B"'«  ^  l/2Z)<=i^»» 
with  diam(5)  <  IjM.  We  say  that  1/M  is  the  majority  number  of  A  if  A  has  a 
1/M-majority  set  B,  but  no  1/A'^-majority  set  for  any  N  >  M.  If  5  C  A  is  a  1/M- 
majority  set  for  all  M  >  0,  then  necessarily  is  a  singleton,  and  is  unique  (we  call 
B  a  0-majority  set).  In  this  case,  we  say  that  A  has  majority  number  0.  We  say  that 
5  is  a  majority  set  for  {A,  W)  if  5  is  a  £-majority  set  for  some  e  >  0. 
Let 

MSET{A,  W)  =  U{B  C  A  I     is  an  e-majority  set}, 

and  let  MA  J(A,  W)  =  least  element  of  MSET{A,  W),  where  e  is  the  majority  number 
of  A. 

We  verify  some  properties  of  majority  sets  which  will  be  useful  in  the  next  few 
theorems. 

Lemma  6.3.8.  Let  A  =  {aj, . . .  ,ak}  and  W  =  {iwi, . . .  ,iujt}  be  finite  sets  of  ratio- 
nals. If  Bi,B2  are  majority  sets  for  (A,  W),  then  5i  D  ^2  7^  0. 
Proof.  Suppose  BinB2  =  0.  Then, 

=  E  "''^  E      ^^"H  i.n    =  I  ^ 

k  k  .  t  -  >^ 

<  l/2j2^i  +  1/2E^'  ^'  ^ 

=  E 

a  contradiction.  □ 


Lemma  6.3.9.  Let  A  and  W  be  as  above.  If  1/M  is  the  majority  number  of  A,  then 

d\a.m{MSET{A,W))<2/M.  ,,,, 

Proof.  Let  61,62  e  MSET{A,W).  Then  there  are  1/M-majority  sets  Bi  and  B2 
with  61  €  Bi  and  62  G  ^2-  By  6.3.8,  there  is  a  c  €  5i  D  B2.  Then 

|6i  -  62I  <  |6i  -  cl  +  |c  -  62I  <  1/M  +  1/M  =  2/M, 
as  desired.  Q 

Theorem  6.3.10.  Let  C  c  QREC.  If |  C  e  NVoo[A]})  >  0,  then  C  €  NVoo. 
Proof.  Let  <S  =  {A  |  C  €  iVVoo[A]},  and  suppose  fx{S)  >  0  (note  that  S  is  mea- 
surable). Let  Sn  =  {A  \  C  e  NVoo[A]  via  $f }.  Since  5  =  [j^Sr,,  there  is  an  e 
with  /i(<Se)  >  7a  for  some  positive  rational  a.  Now,  there  is  an  open  set  Q  contain- 
ing «?e,  with  fi{Q  —  ^e)  <  a.  Furthermore,  since  Q  is  open,  there  is  a  basis  element 
B  =  ULi  ^i^k)  contained  in  g  with  fi{g  -  B)  <  a. 

Now,  suppose  f  £  C,  and  fix  M  >  0.  Then  we  may  stratify  Se  according  to 
how  soon  /  is  inferred  to  within  1/M,  as  follows.  For  each  N  ^  uj,  let 

SeAM)  =  {BeSe\yn>N  |$f  ((/(90), . . .  ,  /(9n-i)))  -  /(gn)|  <  1/M}. 

Note  that  {«Se,N(M)}^^o  is  nondecreasing,  with  Se  =  [jj^Se,N{M).  Thus,  for  any 
/  E  C,  M  >  0,  there  is  an  Nj{M)  such  that  for  all  N  >  Nj{M),  //(5e -5e,yv(M))  <  a. 
Hence,  for      >  Nj{M),  fi{Se,NiM)  H  B)  >  4a,  and  n{B  -  Se,N{M))  <  2a. 

We  now  give  the  construction  of  G  to  NVoo-infer  C.  Let  /  €  QREC.  Then  G 

operates  on  /  as  follows. 

Stage  n: 

Step  1:  Compute  An  =  {v\3BeB^f{{f{qo),...J{qr..i)))  =  v}. 
Step  2:  Enumerate      as  Ui, . . .  , ujt. 
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Step  3:  Compute  the  weight  set  W{An)  for  A„: 

Wi  =  ix{BeB\  ^fiifiqo), . . .  ,/(gn-i)))  =  Vi). 

Step  4:  Output  MA J{An,W{Ar,)). 

End  of  Construction. 

Since  ,6  is  a  finite  union  of  intervals,  only  a  finite  number  of  computations  are 
made  in  step  1  at  any  stage  n.  Not  all  of  the  $f  necessarily  NVoo-infer  C,  but  every 
computation  in  step  1  terminates,  since  $P  is  categorical.  Thus,  the  algorithm  is 
well-defined,  and  total  for  any  input  /.  Now,  if  /  e  C  and  M  >  0,  then  for  all  stages 
n  >  Nf{M),  we  have 

Se,N,mW  C{BeB\  |$f  ((/(go), . . .  ,  fiqn-i)))  -  f{n)\  <  1/M}, 

so  that 

K{B  €  B  I  |$f  ((/(go), . . .  ,  /(gn-i)))  -  f{n)\  <  1/M})  >  4a. 

Thus,  since  for  each  n  >  Nj{M),  fi{B  -  <Se,„(M))  <  2a,  so  that  the  majority  number 
of  An  is  no  greater  than  2/M.  So,  by  the  previous  lemmas,  G  outputs  values  within 

4/M  of /(n)  after  stage  Nf{M).  □ 

Definition  6.3.11.  The  rational  majority  amalgamation  {RMAM)  procedure  is  de- 
fined, for  arbitrary  input  x,  from  parameters  I  =  {ei,  62, . . .  ,  e^}  and  W  =  {tui,  tuj, 
. . .  ,  Wk}  as  follows. 

Input  x: 

Step  1:  Dovetail  the  computations  of      {x), . . .  ,  ^ski^)  until  indices  Cfei, . . .  ,  efc„ 
are  found  such  that  every  <^ek-  {^)  4-,  ^^i^ 

n  k 

(if  no  such  indices  are  found,  then  RMAM{X^  W){x)  t).  *  r  *'  ■ 


Step  2:  Enumerate  the  set  {v  \  3i  $e^.  (x)  =  u}  as  {ui, . . .  ,  v{). 

Step  3:  For  each  i  <  I,  let  Ei  =  {ejt^  |  $e*^  (x)  =  u,},  and  n  =  J^e^^eEi  '^kj- 

Step  4:  Output  MA J{{vi,...  ,vi},{r I,...  ,ri}). 
End  of  Procedure.  -  '  , 

Theorem  6.3.12.  Let  C  c  QREC.  If |  C  €  BCoo[A]})  >  0,  then  C  €  BCoo- 
Proof.  We  define  S,  <Se,  {<Se,iv(M)}^^o,  Q  and  B  analogously  to  the  sets  of  The- 
orem 6.3.10.  Let  S  =  {A  \  C  e  BCoo[A]},  with  fi{S)  >  0.  For  each  n,  set 
Sn  =  {A  \  C  e  BC[A\  via  $^}.  As  before,  there  is  an  e  with  fJ,{Se)  >  7a  for 
some  positive  rational  a,  an  open  set  Q  containing  Se,  with  fi{Q  —  Se)  <  a,  and  a  basis 
element  B  =  IJLi  -^(^fc)  contained  in  Q  with  -  B)  <  a.  li  f  eC,  we  stratify  «Se 
according  to  how  soon  /  is  inferred,  as  follows.  For  each  N  €.  u;  and  M  >  0,  let 

SeAM)  =  {BeSe\yn>N  ||$f((/(go),...  ,/(9n-i)))  -/IL  < 

so  that  for  fixed  M,  {<Se,N(A^)}^=o     nondecreasing  (note  that  Se  =  Utv 
Thus,  for  any  f  e  C,M  >  0,  there  is  an  Nj{M)  such  that  for  all  N  >  Nf{M), 
fi{Se  -  Se,N{M))  <  a.  Hence,  for  N  >  Nf{M),  fi{Se,NiM)  0  B)  >  4a,  and  fi{B  - 
Se,N{M))  <  2a.  We  construct  G  to  BCoo-infer  C  as  follows. 

Stage  n: 

Step  1:  Compute  .-■ 

V.' 

In  =  {eB,n  I  eB,n  =  $f  ((/(^o),  •  •  •  ,  /(^n-i))),  some  BeB}. 
Step  2:  Compute  VK(/„)  as  in  Theorem  6.3.5. 

Step  3:  Output  an  index  (uniformly  chosen)  of  RMAM(/„,  VK(/„)). 
End  of  Construction. 


■••-''••itt.-  ' 
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Analogously  to  Theorem  6.3.10,  at  every  stage  n,  the  computation  in  step  1 
terminates,  producing  a  finite  set  of  indices  /„.  If  /  G  C,  M  >  0,  then  for  all  stages 
n  >  Nf{M),  we  have 

Se,N,iM){M)c{BeB\\\^f-f\\^<l/M}, 

so  that  for  any  x, 

K{B€B\\^.^Jx)-f{x)\<l/M})>4a. 
Thus,  by  Lemma  6.3.9,  we  have  ||RMAM(/„,  iy(/„))  - /||^  <  3/M  for  any  n  > 


Definition  6.3.13.  Let  f,g  =  ^•e  €  QREC.  We  say  that  g  is  an  EX^-variant  (and 
that  e  is  an  EXoo-index)  of  /  if  ,»  ,        '  ^ ^ 

lim  - /(gn)|  =  0.  '  -^ 

n-»oo 

In  this  case  we  write  g  ~oo  /• 

Theorem  6.3.14.  Let  C  c  QREC.  If  ii{{A  \  C  €  EXoo[A]})  >  0,  then  C  €  EXoo- 
Proof.  We  define  <?,  Sg,  {»?e,N}^=o>  ^  '"•i^d  B  analogously  to  the  sets  of  Theo- 
rem 6.3.6.  Let  S  =  {A\C  e  BC[A]},  ^(S)  >  0.  For  each  n,  set  5„  =  {A  |  C  G 
EXoo[A]  via  $^}.  As  before,  there  is  an  e  with  )u(«Se)  >  7a  for  some  positive  rational  a, 
an  open  set  ^  containing  <Se,  with  —  ^e)  <  a,  and  a  basis  element  B  =  Ufc=i  H^k) 
contained  in  Q  with  —  <  a.  If  /  €  C,  we  stratify  <Se  according  to  how  soon  / 
is  inferred,  as  follows.  For  each  A''  €  w,  let 

5e,N  =  {5  G  <Se  I  3eB  Vn>A^  $f((/(9o),...  ,/(g„_i)))  =  CB  and 

— oo  /}? 

so  that  {<Se,;v}^=o  nondecreasing,  with  Se  =  {J^S^n-  Thus,  for  any  f  £  C, 
there  is  an  Nj  such  that  for  all  N  >  Nj,  fi{Se  —  Se,N)  <  o-  Hence,  for  A'^  >  A/, 
(j,{Se,N  (^B)  >  4a,  and  so,  iJ,{B  —  Se,N)  <  2a. 
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Let  CB.n  denote  $f  ((/(go),  •  •  •  ,  /(9n-i))),  and  let  10  =  ^.  We  construct  G  to 
EXoo-infer  C  as  follows  (let  n  >  0): 

Stage  n: 

^fep      Set  P„  =  2„_i,  and  ^„  =  {B  €  B  |  eB,„  =  eB,„_i}. 
^fep      If  pL{Vn  n  :F„)  <  4a,  let      =       otherwise  let  In  =  Pn- 
5<ep  5:  Compute  W{In)  as  above. 

5iep  4'  Output  an  index  (uniformly  chosen)  of  RMAM{In,  W{In)). 

End  of  Construction. 

Analogously  to  Theorem  6.3.6,  at  every  stage  n,  the  computation  in  step  1 
terminates,  producing  a  finite  set  of  indices  /„.  If  /  €  C,  then  for  all  stages  n  >  Nj, 
we  have  Se,Nf  C  ^n,  so  that  ti{!Fn)  >  4a.  Fix  n  >  Nj.  Now,  for  each  M,K  >  0, 
define 

TxiM)  =  {Be  Se,n  \yk>K  |$f  (g,)  -  f{qk)\  <  1/M}. 

For  fixed  M,  {7k(M)}^^i  is  an  increasing  sequence,  with  U/c=i'^(^)  —  ^e,n- 
Thus,  there  is  a  K/iM)  such  that  for  all  k  >  Kj{M),  fi{TkiM))  >  4a.  Thus, 
fx{B  -  Tk{M))  <  2a  for  k  >  Kj,  so  that  for  these  fc, 

\RMAM{Fr.,W{Fr:)){qk)-f{qk)\<2lM.     '  r:,.       V  : 

s 

Thus,  RMAM{Fn,  W{Fn))  c^oo  /• 

We  claim  that  for  n  >  Nj,  the  consequent  of  step  2  is  invoked  at  most  once,  and 
that  lim„  G{(f  (go),  ■  ■  ■  ,  /(?«)))  exists  and  is  an  EXoo-index  of  /.  If  the  consequent  of 
step  2  is  invoked  after  stage  Nj,  let  n  be  the  stage  of  the  first  such  invocation.  Then 
at  this  stage,  we  set  1^  =  Tn.  Since  n  >  A^/,  for  all  n  >  n,  ii{Tn  H  T^)  >  4a,  so  that 
the  consequent  of  step  2  is  not  invoked  at  stage  n,  whence  Xn  =  for  all  n  >  n. 
But  then,  as  noted  previously,  RMAM(E„,  W^(F„))  ~oo  /,  so  G  EX-infers  /. 
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Suppose,  on  the  other  hand,  that  the  consequent  of  step  2  is  never  invoked 
at  any  stage  n  >  Nj.   Then  there  is  a  set  of  oracles  I  with  >  4a  so  that 

for  all  n  >  Nf,  In  =  1,  so  that  for  all  n  >  Nj,  M  outputs  a  canonical  index  of 
RMAM(/,  W{I)),  where  /  denotes  Ind{I).  We  claim  that  RMAM{I,  W{I))  ~oo  /, 
Fix  n  >  Nf,M  >  0,  and  let  define  {Tk{M)},  and  Kf{M)  as  above.  By  hypothesis, 
/i(J^„nJ)  >  4a.  Andby  choiceofn,for  A:  > /i:/(M),/x((^-rfc(M))nJ)  <2a.  Thus, 
for  these  k,  \RMAM{I,  W{I)){qk)- f{qk)\  <  4/M.  Thus,  RMAM{I,  W{I))  ~oo  /•  □ 

The  above  results  give  us  an  analogue,  for  each  inference  class  discussed,  of 
the  theorem  of  Sacks  mentioned  above: 

Corollary  6.3.15.  Let  1  e  {NV,  EX,  BC,  iVVoo,  EX^,  BCoo}-  If  A  is  not  I-trivial, 
then  the  I-cone  above  A  has  measure  zero. 

Proof.  Suppose  fi{{B  \  I[A]  C  I[B]})  >  0.  Then,  for  any  C  €  X[A],  fi{{B  \  C  € 
I[B]})  >  0,  whence  C  G  I.  □ 


CHAPTER  7 
SUMMARY  AND  CONCLUSIONS 

7.1    Summary  and  Open  Problems 

We  have  utilized  the  standard  metric  on  Q  to  extend  the  basic  notions  of  in- 
ductive inference  in  a  natural  way,  allowing  us  to  infer  a  larger  class  of  functions, 
and  in  particular,  to  infer  classes  of  continuous  functions.  We  have  explored  the  re- 
lationships among  these  new  notions  of  approximate  inference,  as  well  as  between 
these  notions  and  the  basic  notions  iVV,  EX,  and  BC.  Specifically,  we  gave  precise 
inclusions  between  the  new  inference  notions  and  those  in  the  standard  inference  hier- 
archy. We  also  explored  weaker  notions  of  approximate  inference,  leading  to  inference 
hierarchies  analogous  to  the  EX"  and  BC"  hierarchies.  Oracle  inductive  inference  was 
also  considered,  and  we  gave  sufficient  conditions  under  which  approximate  inference 
from  a  generic  oracle  G  is  equivalent  to  approximate  inference  with  only  finitely  many 
queries  to  G.  Whether  these  conditions  are  also  necessary  remains  an  open  question. 
Finally,  we  employed  approximation  techniques  from  topology  and  analysis  to  obtain 
a  new  result  regarding  triviality  in  oracle  inductive  inference  classes. 

We  have  only  begun  to  explore  the  area  of  approximate  inductive  inference. 
In  the  remaining  sections,  we  offer  some  ideas  for  futher  research  in  this  field. 

7.2    Stability  -.  >  ./ 

Recall  that  the  standard  inference  hierarchy  is  linear,  that  is 

NV=  PEXCEXCBC, 

but  the  analogous  relation  does  not  hold  for  the  approximate  inference  classes.  In 
particular,  it  is  the  class  iVVoo  which  "ruins"  the  analogy.  We  wish  to  explore  ways 


to  redefine  the  notion  of  iVVoo  to  remedy  this  situation.  To  show  that  NV  C  PEX, 
at  stage  A^,  one  uses  the  outputs  from  an  iW-machine  M  at  stages  n  >  N  as  the 
inputs  to  the  following  stages  to  create  a  function  to  use  as  a  guess  for  the  input 
function  at  the  stage  N.  We  recall  from  chapter  3  the  following  definition  (modified 
to  accommodate  functions  on  /q),  which  formalizes  this  procedure. 

Definition  7.2.1.  For  any  M,a  =  (ao, . . .  ,  Om-i),  we  define  Sm,^  on  Iq  by  recursion 

as  follows 

{M((ao,...,a„_i)),  if  n<m, 

M{{ao,...  ,am-uSM,<jiqm),--  -  , 
SMAQn-i)),  otherwise. 

We  now  introduce  our  first  notion  of  stability  for  NV. 

Definition  7.2.2.  We  say  that  C  is  iVVoo-stable  if  there  is  an  M  which  JVVoo-infers 

C,  and  for  all  /  €  C  there  is  a  stage  L  so  that  for  all  stages  I  >  L, 

lim  \SM,a,{qn)  -  /(9n)|  =  0, 

n-¥oo 

where  ai  =  {/(qo),  ■  ■  •  >/(9/))-  Denote  by  SNVoo  the  class  of  all  such  C. 

It  is  then  easy  to  see  the  following. 
Proposition  7.2.3.  SNVoo  =  PBCoo. 

This  does  not  quite  get  us  the  desired  inclusion,  however,  since  PEXqo  $ 
PBCoo-  We  need  an  even  stronger  notion  of  stability  to  achieve  this. 

Definition  7.2.4.  We  say  that  C  is  iVVoo-superstable  if  there  is  an  M  which  5NVoo- 
infers  C,  and  for  all  /  G  C  there  is  a  stage  L  so  that  for  all  stages  k,l  >  L, 

Denote  by  SSNVoo  the  class  of  all  such  C. 

We  then  obtain  the  desired  result: 
Proposition  7.2.5.  SSNVoo  =  PEXoo-  ^ 
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7.3    Other  Notions  of  Approximate  Inference 

Another  scheme  for  defining  notions  of  approximate  inference  is  one  in  the 
style  of  Egorov's  Theorem.  We  desire  our  inference  method  to  get  "close"  to  the 
input  function,  except  on  a  set  of  size  e. 

Definition  7.3.1.  We  say  that  M  ENVs-infers  C  if  there  is  a  computable  set  E  C 

[0, 1]  with  i^{E)  <  £  such  that  for  each  /  €  C, 

lim  |M((/(go), . . .  ,/(9n.-i)))  - /(9nJ|  =  0, 

fc— »-oo 

where  {quk}  denotes  the  subsequence  of  {g„}  given  by  the  elements  Qn  ^  E. 
Definition  7.3.2.  We  say  that  M  EEX^-infers  C  if  there  is  a  computable  set  E  C 
[0, 1]  with  fJ.{E)  <  e  such  that  for  each  f  EC,  there  is  an  index  e  (of  a  total  function) 
and  a  stage  A''  such  that  for  all  n>  N,  M{{f{qo), . . .  ,f{qn)))  =  e,  and 

lim  |$e(9nj-/(gnj|  =  0, 

where  {^n^}  denotes  the  subsequence  of  {g„}  given  by  the  elements  g„  ^  E. 
Definition  7.3.3.  We  say  that  M  EBC^-infers  C  if  there  is  a  computable  set  E  C 
[0, 1]  with  fi{E)  <  e  such  that  for  each  f  EC, 

lim  ||$M((/(,o)  /(9n-i)))  -  fWt  =0' 

A:— >oo 

where  ||  •  ||^  indicates  that  the  infimum  is  taken  over  q  E  Iq  —  E. 

These  definitions  yield  inference  hierarchies  distinct  from  the  previous  ones. 

7.4    Inference  of  Non-recursive  Functions 

As  noted  in  the  introduction,  the  criteria  for  sucessful  inference  in  the  standard 
classes  NV,  EX,  and  BC  limit  us  to  inference  of  recursive  functions.  In  contrast,  the 
ideas  of  approximate  inference  allow  us  to  extend  the  notion  of  inductive  inference 
to  include  non-recursive  functions.  For  example,  the  linear  interpolation  procedure 
works  to  NVoo-  or  BCoo-infer  the  class  of  all  uniformly  continuous  functions  on  Iq 


74 


(of  which  RUC  is  a  proper  subset).  Now,  if  we  take  our  domain  of  inference  to 
be  the  set  of  all  /  :  Iq  — >  /q,  many  interesting  questions  arise.  Clearly,  not  all 
of  these  functions  are  inferable  by  any  of  the  methods  given.  For  example,  fix  any 
non-computable  irrational  a  in  [0, 1],  then  if  we  define  x  '■  Iq  — ^  by 

,  ,         /  0,   if  g  <  a, 

then  the  singleton  {x}  is  not  JVVoo-  or  B Coo -infer able:  since  x  is  0, 1- valued,  if  it  is 
NVoo  (resp.  BCoo)  inferable,  then  it  is  iVV(resp.  BC)  inferable. 

7.5    Inference  of  Real- valued  Functions 

Slightly  generalizing  the  input  procedure  for  the  approximate  inference  classes 
will  allow  us  to  further  extend  the  domain  of  inference  to  include  all  real-valued  func- 
tions (for  an  alternate  formulation  see  Apsitis,  Freivalds  and  Smith  [2]).  The  actual 
machinery  is  only  slightly  changed.  Suppose  that  /  maps  [0, 1]  into  [0, 1].  We  assume 
some  fixed  enumeration  {qn}  of  the  elements  of  Iq,  in  which  each  rational  appears 
infinitely  often,  and  make  guesses  based  on  finite  sequences  of  pairs  of  rationals  {q,  r), 
where  r  represents  a  rational  approximation  f{q)  for  the  input  function  /.  We  may 
then  use  these  "updates"  to  the  approximation  of  /(x)  for  each  x  to  try  to  iVVoo- 
or  BCoo-infer  /.  In  fact,  the  usual  linear  interpolation  procedure,  modified  to  use  at 
each  stage  the  latest  approximations  given,  works  to  infer  the  class  of  all  continuous 
functions  mapping  [0, 1]  into  [0, 1].  Since  the  continuous  functions  on  [0, 1]  are  deter- 
mined by  their  values  on  Iq  =  [0,  l]nQ,  we  only  need  to  approximate  f{q)  for  rational 
q.  But  q  appears  infinitely  often  in  our  enumeration  {qn},  say  as  the  subsequence 
{qnk}i  so  the  interpolation  procedure  M  will  produce  approximations  ak  =  M{{f{qo), 
...  ,  /(g„^_i))),  whence  f{q)  =  limfc  ajt. 

7.6    Inductive  Feature  Extraction 

We  may  wish,  for  instance,  to  compute  /'or  //  from  /.  All  of  the  induc- 
tive inference  paradigms,  standard  as  well  as  approximate,  can  be  used  as  "feature 
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extraction"  tools  to  compute  in  this  manner.  For  example,  the  standard  inference 
classes  can  be  used  to  compute  "formal"  derivatives  of  the  class  of  polynomials  over 
N.  With  the  techniques  of  approximate  inference  we  can  do  a  bit  more.  For  ele- 
ments of  QREC,  we  can  compute  approximations  to  (true)  derivatives.  For  example, 
if  C  C  RUC  is  a  class  of  functions  /  for  which  /'  is  continuous,  we  can  use  linear 
interpolation,  along  with  the  mean  value  theorem,  to  construct  a  machine  M  which, 
upon  input  f{qo),f{qi),---  outputs  functions  which  approximate  /'  in  the  JVVoo  or 
BCoo  sense.  Note  that  it  is  not  necessarily  the  case  that  /'  is  an  element  of  QREC, 
or  that  its  range  is  contained  in  /q.  Thus,  this  type  of  feature  extraction  provides 
a  natural  setting  in  which  to  extend  the  domain  of  functions  under  consideration  to 
ones  which  are  non-recursive,  and  real-valued. 


^.^^^  REFERENCES 


[1]  D.  Angluin  ajid  C.  H.  Smith,  Inductive  inference:  theory  and  methods,  Comput- 
ing Surveys  15,  1983,  pp.237-269. 

[2]  K.  Apsitis,  R.  Freivalds  and  C.  H.  Smith,  On  the  inductive  inference  of  real- 
valued  functions,  Proceedings  of  the  Eighth  Annual  Conference  on  Computa- 
tional Learning  Theory,  Association  for  Computing  Machinery,  1995,  pp. 170- 
177. 

[3]  Barzdin,  Prognostication  of  automata  and  functions,  Information  Processing  '71, 
North  Holland,  1972,  pp.81-84. 

[4]  L.  Blum  and  M.  Blum,  Toward  a  mathematical  theory  of  inductive  inference, 
Information  and  Control  28,  1975,  pp.  125-155. 

[5]  J.  Case,  S.  Jain  and  A.  Sharma,  On  learning  limiting  programs,  International 
Journal  of  Foundations  of  Computer  Science  3,  1992,  pp. 93-1 15. 

[6]  J.  Case  and  C.  Smith,  Comparison  of  identification  criteria  for  machine  inductive 
inference.  Theoretical  Computer  Science  25,  1983,  pp.  193-220. 

[7]  D.  Cenzer  and  J.  Remmel,  H^-classes  in  analysis,  (to  appear). 

[8]  J.  Feldman,  Some  decidability  results  on  grammatical  inference  and  complexity. 
Information  and  Control  20,  1972,  pp.244-262. 

[9]  L.  Fortnow,  W.  Gasarch,  S.  Jain,  E.  Kinber,  M.  Kummer,S.  Kurtz,  M.  Pleszkoch, 
T.  Slaman,  R.  Solovay,  F.  Stephan,  Extremes  in  the  degrees  of  inferability 
(Manuscript). 

[10]  E.  M.  Gold,  Language  identification  in  the  limit.  Information  and  Control  10, 
1967,  pp.447-474. 

[11]  P.  Halmos,  Measure  Theory,  Van  Nostrand,  New  York,  1950. 

[12]  L.  Harrington,  unpublished  manuscript. 

[13]  C.  Jockusch,  Degrees  of  generic  sets,  Recursion  Theory:  Its  Generalizations  and 
Applications,  Cambridge  University  Press,  New  York,  1980,  pp. 110-139. 

[14]  M.  Kummer  and  F.  Stephan,  On  the  structure  of  degrees  of  inferability,  PrcH 
ceedings  of  the  Sixth  Annual  Conference  on  Computational  Learning  Theory, 
Morgan  Kaufmann,  San  Mateo,  CA,  1993,  pp. 117-126. 

[15]  J.  Munkres,  Topology,  Prentice-Hall,  Englewood  Cliffs,  NJ,  1975. 

[16]  P.  Odifreddi,  Classical  Recursion  Theory,  Vol.  I,  North-Holland,  Amsterdcim, 
1989. 


77 


[17]  P.  Odifreddi,  Inductive  Inference  (Manuscript). 

[18]  M.  B.  Pour-El  and  J.  I.  Richards,  Recursively  Enumerable  Sets  and  Degrees, 
Springer- Verlag,  Berlin,  1989. 

[19]  K.  Popper,  The  Logic  of  Scientific  Discovery,  Hutchinson,  London,  1950. 

[20]  H.  Royden,  Real  Analysis,  Macmillan,  New  York,  1968. 

[21]  H.  Rogers,  Theory  of  Recursive  Functions  and  Effective  Computability,  McGraw 
Hill,  New  York,  1967. 

[22]  J.  Royer,  Inductive  inference  of  approximations,  Information  and  Control  70, 
1986,  pp.156-178. 

[23]  G.  Sacks,  Degrees  of  Unsolvability,  Springer- Verlag,  Berlin,  1963. 

[24]  R.  I.  Soare,  Recursively  Enumerable  Sets  and  Degrees,  Springer- Verlag,  Berlin, 
1987. 


BIOGRAPHICAL  SKETCH 


William  Moser  was  born  in  Boston,  Massachusetts,  on  March  17,  1962.  He 
received  bachelor's  degrees  in  mathematics  and  computer  science  from  the  University 
of  Massachusetts  at  Amherst  in  1985.  After  working  for  three  years  as  a  program- 
mer/analyst for  M.I.T.  Lincoln  Laboratory,  he  returned  to  the  University  of  Mas- 
sachusetts, receiving  an  M.S.  in  mathematics  in  1991.  In  1996,  he  received  a  Ph.D. 
in  mathematics  from  the  University  of  Florida,  Gainesville,  FL,  under  the  direction 
of  Dr.  Douglas  Cenzer. 


i- 


78 


I  certify  that  I  have  read  this  study  and  that  in  my  opinion  it  conforms  to 
acceptable  standaxds  of  scholarly  presentation  and  is  fully  adequate,  in  scope  and 
quality,  as  a  dissertation  for  the  degree  of  Doctor  of  Philosophy. 

DouglcLS  il)  Cenzer,  Chairman 
Professor  of  Mathematics 

I  certify  that  I  have  read  this  study  and  that  in  my  opinion  it  conforms  to 
acceptable  standaxds  of  scholarly  presentation  and  is  fully  adequate,  in  scope  and 
quality,  as  a  dissertation  for  the  degree  of  Doctor  of  Philosophy. 


(Joseph  (Glover 

rofessor  of  Mathematics 

I  certify  that  I  have  read  this  study  and  that  in  my  opinion  it  conforms  to 
acceptable  stcindards  of  scholarly  presentation  and  is  fully  adequate,  h^  scope  and 
quality,  eis  a  dissertation  for  the  degree  of  Doctor^fPhilosopl 


William X  ^^itchell 
Professor  of  Mathematics 

I  certify  that  I  have  read  this  study  and  that  in  my  opinion  it  conforms  to 
acceptable  standards  of  scholarly  presentation  and  is  fully  eidequate,  in  scope  and 
quality,  eis  a  dissertation  for  the  degree  of  Doctor  of  Philosophy. 


Rick  L.  Smith 

Associate  Professor  of  Mathematics 


I  certify  that  I  have  read  this  study  and  that  in  my  opinion  it  conforms  to 
acceptable  standards  of  scholcirly  presentation  and  is  fully  adequate,  in  scope  and 
qucdity,  as  a  dissertation  for  the  degree  of  Doctor  of  Philosophy. 


N.  Wilson 
stant  Professor  of  Computer  and 


Information  Sciences 


This  dissertation  was  submitted  to  the  Graduate  Fa<;ulty  of  the  Department  of 
Mathematics  in  the  College  of  Liberal  Arts  and  Sciences  and  to  the  Graduate  School 
and  was  accepted  as  partial  fulfillment  of  the  requirements  for  the  degree  of  Doctor 
of  Philosophy. 


August  1996 


Dean,  Graduate  School 


